{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Appendix B: Combining queries\n",
    "\n",
    "In this example, we'll walk through the process of building a more complex query for the MSMARCO Document dataset. This assumes that you are familiar with the other query tuning notebooks. We'll be using the `cross_fields` and `best_fields` queries and the optimal parameters found as the foundation on which we build a more complex query.\n",
    "\n",
    "As with the previous notebook and in accordance with the MSMARCO Document ranking task, we'll continue to use MRR@100 on the `dev` dataset for evaluation and comparison with other approaches."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%load_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import importlib\n",
    "import os\n",
    "import sys\n",
    "\n",
    "from elasticsearch import Elasticsearch\n",
    "from skopt.plots import plot_objective"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# project library\n",
    "sys.path.insert(0, os.path.abspath('..'))\n",
    "\n",
    "import qopt\n",
    "importlib.reload(qopt)\n",
    "\n",
    "from qopt.notebooks import evaluate_mrr100_dev, optimize_query_mrr100\n",
    "from qopt.optimize import Config"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# use a local Elasticsearch or Cloud instance (https://cloud.elastic.co/)\n",
    "# es = Elasticsearch('http://localhost:9200')\n",
    "es = Elasticsearch('http://35.246.195.44:9200')\n",
    "\n",
    "# set the parallelization parameter `max_concurrent_searches` for the Rank Evaluation API calls\n",
    "# max_concurrent_searches = 10\n",
    "max_concurrent_searches = 30\n",
    "\n",
    "index = 'msmarco-document'\n",
    "template_id = 'combined'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Combining `cross_fields` and `best_fields`\n",
    "\n",
    "Based on previous tuning, we have the following optimal parameters for each `multi_match` query type."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "cross_fields_params = {\n",
    "    'operator': 'OR',\n",
    "    'minimum_should_match': 50,\n",
    "    'tie_breaker': 0.25,\n",
    "    'url|boost': 1.0129720302556104,\n",
    "    'title|boost': 5.818478716515356,\n",
    "    'body|boost': 3.736613263685484,\n",
    "}\n",
    "\n",
    "best_fields_params = {\n",
    "    'tie_breaker': 0.3936135232328522,\n",
    "    'url|boost': 0.0,\n",
    "    'title|boost': 8.63280262513067,\n",
    "    'body|boost': 10.0,\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We've seen the process to optimize field boosts on two different `multi_match` queries but it would be interesting to see if combining them in some way might actually result in even better MRR@100. Let's give it a shot and find out.\n",
    "\n",
    "Side note: Combining queries where each sub-query is always executed may improve relevance but it will hurt performance and the query times will be quite a lot higher than with a single, simpler query. Keep this in mind when building complex queries for production!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'cross_fields|operator': 'OR',\n",
       " 'cross_fields|minimum_should_match': 50,\n",
       " 'cross_fields|tie_breaker': 0.25,\n",
       " 'cross_fields|url|boost': 1.0129720302556104,\n",
       " 'cross_fields|title|boost': 5.818478716515356,\n",
       " 'cross_fields|body|boost': 3.736613263685484,\n",
       " 'cross_fields|boost': 1.0,\n",
       " 'best_fields|tie_breaker': 0.3936135232328522,\n",
       " 'best_fields|url|boost': 0.0,\n",
       " 'best_fields|title|boost': 8.63280262513067,\n",
       " 'best_fields|body|boost': 10.0,\n",
       " 'best_fields|boost': 1.0}"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def prefix_keys(d, prefix):\n",
    "    return {f'{prefix}{k}': v for k, v in d.items()}\n",
    "\n",
    "# prefix key of each sub-query\n",
    "# add default boosts\n",
    "all_params = {\n",
    "    **prefix_keys(cross_fields_params, 'cross_fields|'),\n",
    "    'cross_fields|boost': 1.0,\n",
    "    **prefix_keys(best_fields_params, 'best_fields|'),\n",
    "    'best_fields|boost': 1.0,\n",
    "}\n",
    "all_params"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Baseline evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Evaluation with: MRR@100\n",
      "Score: 0.3083\n",
      "CPU times: user 1.97 s, sys: 686 ms, total: 2.66 s\n",
      "Wall time: 4min 32s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "_ = evaluate_mrr100_dev(es, max_concurrent_searches, index, template_id, params=all_params)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Query tuning\n",
    "\n",
    "Here we'll just tune the boosts for each sub-query. Note that this takes twice as long as tuning individual queries because we have two queries combined."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimizing parameters\n",
      " - metric: MRR@100\n",
      " - queries: data/msmarco-document-sampled-queries.1000.tsv\n",
      " - queries: data/msmarco/document/msmarco-doctrain-qrels.tsv\n",
      " - iteration 1 scored 0.2805 with: {'cross_fields|boost': 3.896685922580469, 'best_fields|boost': 2.650422982146484}\n",
      " - iteration 2 scored 0.2812 with: {'cross_fields|boost': 0.7870601079131091, 'best_fields|boost': 0.8510910404952616}\n",
      " - iteration 3 scored 0.2812 with: {'cross_fields|boost': 3.250176544662946, 'best_fields|boost': 3.425823541294568}\n",
      " - iteration 4 scored 0.2798 with: {'cross_fields|boost': 4.014420699567866, 'best_fields|boost': 1.7532333149891575}\n",
      " - iteration 5 scored 0.2741 with: {'cross_fields|boost': 2.7888523112917554, 'best_fields|boost': 0.37349976261196355}\n",
      " - iteration 6 scored 0.2789 with: {'cross_fields|boost': 1.0211402631272501, 'best_fields|boost': 4.935530516183548}\n",
      " - iteration 7 scored 0.2789 with: {'cross_fields|boost': 1.8255603541324559, 'best_fields|boost': 3.486062377887463}\n",
      " - iteration 8 scored 0.2740 with: {'cross_fields|boost': 4.497190799085895, 'best_fields|boost': 0.6340397795670584}\n",
      " - iteration 9 scored 0.2810 with: {'cross_fields|boost': 1.5779546258471802, 'best_fields|boost': 1.8404702184967972}\n",
      " - iteration 10 scored 0.2810 with: {'cross_fields|boost': 4.8526018215319935, 'best_fields|boost': 2.7622315541562603}\n",
      " - iteration 11 scored 0.2802 with: {'cross_fields|boost': 3.5532712052074915, 'best_fields|boost': 1.8953710215598645}\n",
      " - iteration 12 scored 0.2798 with: {'cross_fields|boost': 3.731302823701837, 'best_fields|boost': 4.987587383054652}\n",
      " - iteration 13 scored 0.2758 with: {'cross_fields|boost': 2.4645378742324544, 'best_fields|boost': 0.46439094045581114}\n",
      " - iteration 14 scored 0.2790 with: {'cross_fields|boost': 0.2986054576725506, 'best_fields|boost': 1.0894228468949338}\n",
      " - iteration 15 scored 0.2782 with: {'cross_fields|boost': 1.4445863095010816, 'best_fields|boost': 4.18343122346622}\n",
      " - iteration 16 scored 0.2808 with: {'cross_fields|boost': 4.999608062630131, 'best_fields|boost': 4.070685933073027}\n",
      " - iteration 17 scored 0.2805 with: {'cross_fields|boost': 5.0, 'best_fields|boost': 3.3981488280472427}\n",
      " - iteration 18 scored 0.2739 with: {'cross_fields|boost': 0.06305474916625088, 'best_fields|boost': 0.00709268279107056}\n",
      " - iteration 19 scored 0.2781 with: {'cross_fields|boost': 0.0, 'best_fields|boost': 2.2504561133876115}\n",
      " - iteration 20 scored 0.2808 with: {'cross_fields|boost': 5.0, 'best_fields|boost': 3.1224424489199123}\n",
      " - iteration 21 scored 0.2808 with: {'cross_fields|boost': 4.9606046747383505, 'best_fields|boost': 4.996794505598459}\n",
      " - iteration 22 scored 0.2783 with: {'cross_fields|boost': 0.08243871186606967, 'best_fields|boost': 2.960988317746084}\n",
      " - iteration 23 scored 0.2783 with: {'cross_fields|boost': 0.0032683432995433974, 'best_fields|boost': 4.96308585491223}\n",
      " - iteration 24 scored 0.2809 with: {'cross_fields|boost': 5.0, 'best_fields|boost': 4.398771898333286}\n",
      " - iteration 25 scored 0.2707 with: {'cross_fields|boost': 4.9258372188450315, 'best_fields|boost': 0.0113675716399686}\n",
      " - iteration 26 scored 0.2782 with: {'cross_fields|boost': 0.01735658022776088, 'best_fields|boost': 3.883207339826874}\n",
      " - iteration 27 scored 0.2788 with: {'cross_fields|boost': 2.533365973395604, 'best_fields|boost': 4.985769547678409}\n",
      " - iteration 28 scored 0.2791 with: {'cross_fields|boost': 4.9902174948096825, 'best_fields|boost': 1.7179977225598861}\n",
      " - iteration 29 scored 0.2803 with: {'cross_fields|boost': 5.0, 'best_fields|boost': 3.7029463719898907}\n",
      " - iteration 30 scored 0.2783 with: {'cross_fields|boost': 0.00027091626548514386, 'best_fields|boost': 1.5649612427277535}\n",
      "Best score: 0.2812\n",
      "Best params: {'cross_fields|boost': 0.7870601079131091, 'best_fields|boost': 0.8510910404952616}\n",
      "Final params: {'cross_fields|operator': 'OR', 'cross_fields|minimum_should_match': 50, 'cross_fields|tie_breaker': 0.25, 'cross_fields|url|boost': 1.0129720302556104, 'cross_fields|title|boost': 5.818478716515356, 'cross_fields|body|boost': 3.736613263685484, 'cross_fields|boost': 0.7870601079131091, 'best_fields|tie_breaker': 0.3936135232328522, 'best_fields|url|boost': 0.0, 'best_fields|title|boost': 8.63280262513067, 'best_fields|body|boost': 10.0, 'best_fields|boost': 0.8510910404952616}\n",
      "\n",
      "CPU times: user 26.7 s, sys: 7.08 s, total: 33.8 s\n",
      "Wall time: 24min 38s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "_, _, final_params_boosts, metadata_boosts = optimize_query_mrr100(es, max_concurrent_searches, index, template_id,\n",
    "    config_space=Config.parse({\n",
    "        'num_iterations': 30,\n",
    "        'num_initial_points': 15,\n",
    "        'space': {\n",
    "            'cross_fields|boost': { 'low': 0.0, 'high': 5.0 },\n",
    "            'best_fields|boost': { 'low': 0.0, 'high': 5.0 },\n",
    "        },\n",
    "        'default': all_params,\n",
    "    }))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWcAAAFQCAYAAABjx7+YAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOy9d5xV1dX///5MoTO0GWDoAiIgygBjb4j6iIpKjC3RWKJiEvM1msSoj8lPTWIeE58YNcWImgexKzYEO4iiYhk6Sm8CAwy9l2Fm/f445+L1OuXeuefOLbPfr9d53XP22WUdhvnMvuusvbbMDIfD4XCkFlnJNsDhcDgc38WJs8PhcKQgTpwdDocjBXHi7HA4HCmIE2eHw+FIQZw4OxwORwrixNnhcDhSECfOASPpPklf+p8/kXRFLfXvkvTrKsp7SJpXS9spknqEnRfHY3sN4/SQ9MNE9O1wOKomJ9kGJANJOWZ2IEHdjwLamllFgvpPBj2AHwLPJNkOh6PBkLEzZ0lXSJojabakJyWNkfRvSZ8Bf5FUJOlTv84rktr47W6U9JVf/pxfdoqkWf4xU1LLasYcD7QApku6JHxWLKmXpLckTZc0VVLfKtoP8e2dDdwQVn64pM/98edIOrSax/6RX2eepKP9tm0lveq3+1TSkbWUV/Ws9wIn+WU31+kH4nA4YsPMMu4ADgcWAfn+dVtgDDAByPbL5gCn+Oe/Bx7wz0uBxv55a//zdeAE/7wFkFPD2DvDzu8Cfu2fTwIO9c+PASZXUWcOcLJ/fh8wzz//O3CZf94IaOqfTwF6hJ0/6p+fHNH2Tv98GDCrlvLvPCswFJiQ7J+rO9zRkI5MnTkPA140s40AZrbZL3/RzCoktcIT3g/88ifwBA08gXxa0uVAyPXxMXC/pBv9djG5RCS1AI4HXpQ0C3gEKIyo09rv+0O/6Mmw29OA/5Z0K9DdzPZUM9Sz/vN+COT5fZ4Y6svMJgPtJOXVUB7XszocjmDIVHGujl1R1DkH+CcwGPjC90/fC1wLNAU+rsolUQtZwFYzKwo7+kXb2MyeAc4D9gBvSBpWXdVarqMZK95ndTgcAZCp4jwZuEhSO/D8q+E3zWwbsEXSSX7Rj4APJGUBXc3sfeBWoBXQQlIvM5trZn8GvgBiEiwz2w4sl3SRb48kDYyosxXYKulEv+iy0D1JPYFlZvYQ8BpwZDVDXeLXPxHY5j/n1FBfkoYCG317qiyv5ll3AFX62R0OR2LIyGgNM/tS0j14glsBzKyi2pXAvyU1A5YBVwPZwFO+20PAQ2a2VdIfJJ0KVAJfAm/WwazLgIcl/RbIBZ4DZkfUuRr4jyQD3gkrvxjvZV85sA74UzVj7JU00+//x37ZXX6fc4Dd/nPXVH5TFc9aCVT4LyrHmNnfYnx2h8MRIzJz+ZzTFUlTgKvMbEWSTXE4HAGTqW4Nh8PhSGsy0q2RaCQdwbejKQD2mdkx9WzKGGBrPY/pcDjqgQYjzpL+A4wAysxsQDx9mdlcoCii/66S3gc64EVJjDazB+MZJxJJTYAPgcZ4P7tx/ovEwJGUDZQAa8xsRCLGcDgc1dOQ3BpjgOEJ7P8A8Csz6w8cC9wgqX/AY+wDhpnZQLw/DsMlHRvwGCF+AcxPUN8Oh6MWGow4+wszNtdase79rzWzGf75Djxh6xzwGGZmO/3LXP8I/I2upC548d6PBd23w+GIjgYjzvWJnyluEPBZAvrO9lcZlgHvmlngYwAPAL/BC6FzOBxJwIlzwPhLtV8CbvIXewSKmVWYWRHQBThaUlz+80gkhfzy04Ps1+FwxIYT5wCRlIsnzE+b2cuJHMt/Efg+wfvRTwDOk7QCb6HMMElPBTyGw+GoBSfOASFJwOPAfDO7P0FjFPjJjJDUFDgDWBDkGGZ2u5l1MbMewKV42fMuD3IMh8NROw1GnCU9i5fd7TBJqyVdE/AQJ+Dl6BgWlg/57IDHKATe95dcf4Hnc54Q8BgOhyMFcMu3HQ6HIwVpMDNnh8PhSCecODscDkcK4sTZ4XA4UhAnzg6Hw5GCOHF2OByOFKTBibOkUW6M5PfvcDhqpsGJM1AfopMJYzhxdjiSSEMUZ4fD4Uh5UmoRSn5+vvXo0SO4DktLoVOnbxVt2LCBgoKC4MaogkwYoz6eYfr06RvNLLGDOBxpSkrthNKjRw9KSkqC61DyBNqRkkhamWwbHI5UJbPdGtddl2wLHA6Ho05ktjiPHp1sCxwOh6NOZLY4DxmSbAsc1bC3vCLZJjgcKU1K+ZwDZ8aMZFvgiKCy0hg/u5T73l6YbFMcjpQms8XZkVJ8umwT90ycz9w12xjQOS/Z5jgcKU1mi3NhYbItcADbdpfzx4lf8eL01XRu3ZS/XTKQ8wd2JvvGZFvmcKQumS3OLowu6bw1by2/e+1LNu/az8+G9uLG0w6lSW52ss1yOFKezH4heNddybagwbJtdzk3PjuTnzw1g/YtGzP+5yfwm+F9nTA7HFGSUisEi4uLLfBFKCn0fA2FqYs3cMuLc9i4cx83nnYoPxvai5zs784DJE03s+IkmOhwpDyZ7dZw1Cu79x/g3jcXMHbaSnoVNOfRK07giC6tkm2Ww5GWOHF2BELJis386sXZrNy0mx+fcAi/GX6Yc2E4HHGQ2eIcpIvEUSU79x3gr+8sZMwnK+jcuinPXncsx/Vql2yzHI60J7PF2ZEwzIy35q3j7te/Yv2Ovfzw6G7cfnY/WjR2/6UcjiDI7N+k4mL3QjABzPh6C399ZyEfL9lE/8I8Hr58MIO6tUm2WUlBUgfgT0AnMztLUn/gODN7PMmmOdKczBZnR2CYGbNXb+PvkxYzaUEZ7Zo34s5z+/OjY7tXGYnRgBgD/B9wh3+9CHgecOLsiAsnzo4a2bxrP6/OXMMLJatYsG4HrZrmcsuZh3HV8T1o7lwYAPlm9oKk2wHM7IAkl9XJETeZ/dt1553JtiAplO3Yy5Zd5ewpr2BveQVZEs0aZdO8cQ7N/c9mjbKRdLBNZaWxcec+1m7by9ebdzN95RY+W76ZBeu2YwYDu7TijyMHcF5RJ/Ka5Cbx6VKOXZLaAQYg6VhgW3JNcmQCmS3ODWSF4Mad+xg3fTXTV25hzuqtrN++r9Y2WYLGOdkYhhkcqDQqKr/xzzfJzWJI9zbcdFof/uvwDvQrdImKquGXwHigl6SPgQLgwuSa5MgEMlucO3XK6PwaqzbvZvSHy3ihZBX7DlTSM785x/fK54jOreiQ14SmjbJokpNNpcGu/QfYvf8AO/dVsGvfAXbtO8De8gokIUFOluiQ14TCVk0pbNWEPh1a0iinQfuSo8LMZkg6BTgMELDQzMqTbJYjA8hscV67NtkWJAQz4+EPlvLXdxaRJbhgUBdGndKTXgUtkm1avSGpLd6Ltx7ACuBiM9sSUacIeBjIAyqAe8zsef/eVKClX7U98LmZjfTvDQUeAHKBjWZ2Sg123AA8bWZf+tdtJP3AzP4VzJM6Giout0aasXv/AW4ZN4eJc9ZyzpGF/O6c/nRs1STZZtWJeHJrSPoLsNnM7pV0G9DGzG6NqNMHMDNbLKkTMB3oZ2ZbI+q9BLxmZmMltQY+AYab2deS2ptZWQ12zDKzooiymWY2qC7P5XCEyOyZ8+DBybYgUNZs3cM1Y75g0fod3H5WX0ad3PNbL/UaGOcDQ/3zJ4ApwLfE2cwWhZ2XSirD8wkfFGdJecAw4Gq/6IfAy2b2td+uWmH2yZYk82c5krKBRnV7JIfjGzJbnKdPT7YFgbF9bzlX/edz1m3fy5irj+bkPgXJNinZdDCzkN9qHdChpsqSjsYTzaURt0YCk8xsu3/dB8iVNAXP7fGgmY2toeu3gOclPeJfX++XORxxkXRxljQKGAXQrVu3YDsfNSojduA+UFHJ/3tmJss37mLsNUdzfK/8ZJsUFPmSwv1Yo83s4A9M0ntAxyra3RF+YWYmqVr/laRC4EngSjOrjLj9A+CxsOscYAhwGtAUmCbp0/BZeAS34gnyT/3rdyP6czjqhPM5pwG/f/0r/vPxcu753gAuO6Z7ss0JjDh9zguBoWa21hffKWZ2WBX18vBcHn8ys3ER9/KBhUBnM9vrl90GNDWzO/3rx4G3zOzFutjpcNQVFyuV4rxQsor/fLycq47vkVHCHADjgSv98yuB1yIrSGoEvAKMjRRmnwuBCSFh9nkNOFFSjqRmwDHA/OqMkHSCpHclLZK0TNJyScvq+EwOx0GS7tZwVM/67Xv5/etfcWzPtvz2nH7JNifVuBd4QdI1wErgYgBJxcBPzOxav+xkoJ2kq/x2V5nZLP/8Ur+fg5jZfElvAXOASuAxM5tXgx2PAzfjRYK4ZduOwMhst0ZpqbcQJU254ZkZvPvVet656WR65DdPtjmBkwnbVEn6zMyOSbYdjswjs2fO06enrThPWVjGxDlr+eUZfTJSmDOI9yXdB7wMHFw3b2YzkmeSIxPIbHE+77y0fCG4t7yC3702j54Fzbn+lJ7JNsdRM6FZc/g3AMOLnXY46kxmi3Oa8o/JS1i1eQ/PXncsjXPcPnypjJmdmmwbHJmJi9ZIMcp27OXRqcsYWdTJ7cWXBkjqIOlxSW/61/39l5QOR1yklDjv3HeA/Qci1wjEwSOP1F4nxRj9wTLKKyq56fQ+yTbFER1jgLeB0MuNRcBNSbPGkTGklDgv37iLkhWbg+tw1Kjg+qoHNuzYx1OfrWRkUWf3EjB9yDezF/DC7jCzA7iQOkcApJQ4C5i0oLY8M7F0mF5JgR6buoz9Byr5+bDeyTbFET1uJxRHQkgpcW7ROIfJQYpzGrFp5z7GTlvJeQM70bMB5WXOACJ3QhkL/L/kmuTIBFIqWqNl01yWb9zFsg07G5xAPTp1OXsPVLhZc5rhdkJxJIrUEufGOWwHJi8oC0acR4yIv496YPvecp6ctoJzjiikd/uWtdZ3JB9JF1Rzq48kzOzlejXIkXGklDg3ysnisA4tmTS/jGtPCmDxxeuvx99HPfDS9NXs2l/BqJPdgpM04lz/sz1wPDDZvz4VbycVJ86OuEgpnzPAsH7t+WLFZrbtCeCb4bnn1l4nyVRWGk9OW0lR19Yc2aV1ss1xRImZXW1mV+PtM9jfzL5vZt8HDvfLHI64SDlxPq1vew5UGlMXb4i/swkT4u8jwXy8dCPLNu7iiuNcOtA0pWvYjiwA64GAd41wNERSTpwHdWtD62a5TJ7fMKI2xk5bSbvmjTj7iMJkm+KoG5MkvS3pKj8t6UTgvSTb5MgAUsrnDJCdJU49rD3vLyyjotLIzkqvWOVYWL1lN5Pmr+enQ3vRJNfl0EhHzOzn/svBk/yi0Wb2SjJtcmQGKSfOAMP6tueVmWuY+fUWinu0rXtHKZ6R7unPvgbgh26Hk7TGj8xwLwAdgZJybg2Ak/sUkCX4cFGcfucU3tx1b3kFz33+NWf070Dn1k2TbY6jjki6QNJiSdskbZe0Q9L22ls6HDWTkuLcqmkuA7u2ZuqSjfF1dP31wRiUAN79aj1bdpdz+bFu1pzm/AU4z8xamVmembU0s7xkG+VIf1JSnAFO6p3P7FVb2bY7MxdbvTRjNZ1aNeH4XvnJNsURH+vNrNoNYB2OupKy4nzioQVUGkxbFufsOQVZv30vHy7awAWDu2T0C88GQomk5yX9wHdxXFDD6kGHI2pS8oUgwKBurWneKJupizcyfEAdw8zGjw/WqIB4ZeYaKg2+P6RLsk1xxE8esBv4r7Ayw70gdMRJyopzbnYWx/Vqx0fx+J2HDAnOoIAwM8ZNX01x9zYc4nI2pz3+KkGHI3BS1q0BcGLvfFZu2s2qzbvr1kHnzsEaFACzV29jSdlOLnSz5riQ1FbSu36kxLuS2lRRp0jSNElfSpoj6ZKwe1MlzfKPUkmv+uWtJL0uabbfrkbxldRH0iRJ8/zrIyX9NujndTQ8UlucDy0AYOrizPE7j5u+iia5WZx9pFsRGCe3AZPM7FBgkn8dyW7gCjM7HBgOPCCpNYCZnWRmRWZWBEzjGzfEDcBXZjYQGAr8VVKjGux4FLgdKPf7nQNcGu/DORwpLc69CppT2KoJHy0JIM9GCrC3vILXZ69l+OEdyWvicuPEyfnAE/75E8DIyApmtsjMFvvnpUAZUBBeR1IeMAx4NdQMaClJQAtgM3CgBjuamdnnEWU11Xc4oiKlxVkSJ/bO5+Mlm6iorMNqv+uuC96oOJi8oIxte8rdi8Bg6BCWcGgd0KGmypKOBhoBSyNujcSbgYcWjvwD6AeUAnOBX5hZTbsOb5TUi2+2qboQWFtDfYcjKpIuzpJGSSqRVLJhw3dnyCcems+2PeXMW1OHbdlSbIXg+FmlFLRs7GKbvyE/9LP3j2/tyCvpPUnzqjjOD69nZoYvjlUhqRB4Eri6CqH9AfBs2PWZwCy83bSLgH/4s+vquAF4BOgraQ3ezts/qfGpHY4oSHq0hpmNBkYDFBcXf+cX7ITenpB9tGQjA7vGmO94yBCYPj1+IwNg+95yJi8s44dHd3Oxzd+w0cyKq7tpZqdXd0/SekmFZrbWF98q0xj6wjoRuMPMPo24lw8cDXwvrPhq4F5f8JdIWg70BSJdFyEblwGnS2oOZJnZjupsdjhiIekz59rIb9GYfoV5fLK0Di8FZ8wI3qA68va8dew/UMn5RZ2SbUqmMB640j+/EngtsoL/Iu8VYKyZjauijwuBCWa2N6zsa+A0v30HvL0Bl1VnhKR2kh4CpgJTJD3o78btcMRFyoszwHE921GyYgv7DlQk25Q6M352Kd3aNqMo1tm/ozruBc6QtBg43b9GUrGkx/w6FwMnA1eFhc0VhfVxKd92aQD8AThe0ly8KJBbzaymmcFzwAbg+3hivwF4Pr5HczhSwK0RDcf3asd/Pl7OzK+3cmzPGCYlhakRrrZx5z4+WbqJn5zSEy8IwBEvZrYJf4YbUV4CXOufPwU8VUMfQ6soK+Xbq/1qo9DM/hB2/cfweGqHo66kxcz56J5tyRJ8snRTbA1LSxNjUIy8MXctFZXGeQNTb1GMI27ekXSppCz/uBh4O9lGOdKftBDnvCa5DOjcik9jFee77kqIPbEyflYph3VoyWEdWybbFEfwXAc8A+wH9uG5Oa53eZ0d8ZIW4gxwXK92zFy1hT37Y/A733134gyKktVbdlOycgvnuReBGYmfvznLzHLMLNc/b+nyOjviJX3EuWc7yiuMkpWbk21KTEyY461HOPdIJ86ZiDwul/Q7/7qrv+DF4YiLtBHno3q0JSdLTIvVtZFkJs5Zy8CurenWrlmyTXEkhn8BxwE/9K93Av9MnjmOTCFtxLl54xwGdm0d20vBkpLEGRQFKzftYu6abYw4IjWiRhwJ4RgzuwHYC2BmW/CWiTsccZE24gyea2Pumm3s3JceeWUmzvVcGmcd0THJljgSSLmkbL7JrVEA1JSLw+GIivQS517tqKg0vlgepd+5uNqVwfXCxDlrGdStNV3aOJdGBvMQ3irE9pLuAT4C/pRckxyZQFqJ85DubWiUnVW3pdz1zPKNu/iydDvnOJdGRmNmTwO/Af4HLxvdSDN7MblWOTKBtFghGKJJbjaDurVm2rLUfyn4hu/SONuJc0YiqW3YZRlhy8AltTWz9AorcqQcaTVzBs+18WXpdrbtLq+98p13Jt6gapgwZy1DurehU+umSbPBkVCmAyX+5wZgEbDYP0+NVIiOtCb9xLlnO8zg0+VRzJ6TtEJw6YadzF/rXBqZjJkdYmY9gfeAc80s38zaASOAd5JrnSMTSDtxLurWmia5WdHFO3dKzsKPN+Y4l0YD4lgzeyN0YWZvAscn0R5HhpBWPmeAxjnZFHdvy6fR+J3XJme3oIlz11LcvQ0dWzVJyviOeqXU3207lP3uMrwtrhyOuEi7mTN4fucF63awaee+ZJvyHZaU7WDBuh2McLtrNxR+gLdp7Ct4O3gX+GUOR1yk3cwZOJjT+dNlmzmnJhEcPLieLPqGiXPWIcFZzqXRIPCjMn6RbDscmUdazpyP7NKKZo2ymbaslnjnJOwfOHFuKUf1aEuHPOfScDgcdSctxTk3O4ujD2lb+0vBUaNqvh8wi9fvYNH6nS5Kw+FwxE1aijN4IXVLN+yibPve6is9+mj9GYT3IlCCswa4XBoOhyM+0tLnDN5LQYBpyzZxflFqbP80cc5aju7RlvbOpZHxSPo7frKjqjCzG+vRHEcGkrbifHinVrRsksO0pakhzovW72Bx2U7+cP7hyTbFUT8kNx+tI+NJW3HOzhLHHNK25vzOa9bUmz0T5qwlS3Cmc2k0CMzsiWTb4Mhs0tbnDHBi73y+3rybrzftrrpCPUVrmBkT5pRy9CFtad/SuTTqA0ltJb0rabH/2aaKOkWSpkn6UtIcSZeE3TtN0gxJsyR9JKm3X95Y0vOSlkj6TFKPWuwokPS/kt6QNDl0BP28joZHWovzSX0KAJi6ZEPVFc47r17s+LJ0O8s27OK8gcl3rzQgbgMmmdmhwCT/OpLdwBVmdjgwHHhAUmv/3sPAZWZWhLd79m/98muALWbWG/gb8Oda7HgamA8cAtwNrAC+qOtDORwh0lqce+Y3p1OrJkxdlNz8zuNnl5KbLRelUb+cD4RcC08AIyMrmNkiM1vsn5fipfYsCN0GQrtjt+KbJdfh/Y4DTpOkGuxoZ2aPA+Vm9oGZ/RgYVrdHcji+IW19zgCSOOnQAt6Yt5YDFZXkZNf/35rKSuP12aWcfGgBbZq7rePqkQ5mFkqesg7oUFNlf0fsRsBSv+ha4A1Je4DtwLF+eWdgFYCZHZC0DWgHVDcDCOWuXSvpHDyRb1tNXYcjapI+c5Y0SlKJpJING6pxT9TASX3y2bH3ALNXb/vuzUceCcDCmvlixWbWbtvLeUXJyYCX5uSHfvb+8a1VQ5LekzSviuP88HpmZtQQ1iapEHgSuNrMQvv73QycbWZdgP8D7q/jM/xRUivgV8Cvgcf8vh2OuEj6zNnMRgOjAYqLi6v9BauOE3rlI8HUxRsY0j3inVA9rBAcP7uUprnZnNG/xombo2o2mlm1Gz2a2enV3ZO0XlKhma31xbesmnp5wETgDjP71C8rAAaa2Wd+teeBt/zzNUBXYLWkHDyXR7UhQWY2wT/dBpxaXT2HI1aSPnOOlzbNG3Fk51Z8tLiKb501ugrjp7yikjfmruWM/h1o1ijpf+caGuOBK/3zK4HXIitIaoSXLW6smY0Lu7UFaCWpj399Bt5Lvch+LwQm+zPzyL5/43/+XdJDkUecz+ZwJH/mHAQnHVrAwx8sZfvecvKa5NbbuB8t3siW3eWc71wayeBe4AVJ1wArgYsBJBUDPzGza/2yk4F2kq7y211lZrMkXQe8JKkST6x/7N9/HHhS0hJgM3BpNeOHxNwtRnEkhAwR53z+8f4Spi3dxJmH11/ExGuz1tCqaS4nHVpQe2VHoJjZJuC0KspL8F72YWZP8U0S/Mh6r+DNqiPL9wIXRTH+6/7p7sjdtiXV2t7hqI20d2sADOrWhuaNspm6OOKF4ogRCRtz174DvPPVes4+oiONcjLin9FRN26PsszhiImMmDk3ysniuF7tmBrpd3799aobBMDEuWvZvb+CC4d0SdgYjtRF0lnA2UDnCB9zHnAgOVY5MomMmfKd3KeAlZt2s3TDzm8Kzz03YeONK1lNz4LmDO72nVXDjoZBKZ6/eS8wPewYD5yZRLscGULGiPPp/bxQtre/XPdN4YQJ1dSOj2UbdvL5is1cNKQrNS8ec2QqZjYbz5/9sZk9EXa8bGZbkm2fI/3JGHHu1LopA7u25u1562qvHCfjpq8mO0t8f7DLpdGQMbMKoKsfsudwBEpG+JxDnDWgI/e+uYA1W/fQuXXThIxRUWm8NGM1Q/sUuKT6DoDlwMeSxgO7QoVmVtcVhw4HkEEzZ+BgGN3B2fN31w7EzYeLN7B++z4uKnYvAh2Al6tjAt7vUsuww+GIi4yaOR+S35y+HVvy1pfr+PGJh8Do0YEv4X6xZBVtmzdiWF+3XNsBZnZ3sm1wZCYZNXMGb/b8xYrNbNixD66/PtC+N+3cx7tfred7gzq72GYHcDDZ/n0u2b4jaDJOYYYP6IgZvDd/feB9P/Xp15RXGD84ulvgfTvSlqeBBbhk+46AyThx7tuxJd3bNeOtgKM29pZX8OSnKxjWtz2927cItG9HWuOS7TsSQsaJsySGH96RT5ZuZNeLLwfW7/hZpWzcuZ9rTjwksD4dGcG3ku1LGoRLtu8IgIwTZ4BzjiykvMJ4o1Ew2eLMjMc/Wk7fji05vle7QPp0ZAwu2b4jIWRUtEaII7u0ZmCXVlx0/rFYZWXcq/g+WrKRhet38L8XDXQrAh0ASGoC/ATojbe11eNm5pLtOwIjI2fOAFce3wPwhDVeHpu6nIKWjTl3YGHcfTkyhieAYmAucBbw1+Sa48g0MlaczznSE9InPlkRVz8L1m3ng0UbuPK47jTOyQ7AMkeG0N/MLjezR/B2TDkp2QY5MouMFefGOdnMOesiJi0o4+tNu+vUh5lxz8T55DXJ4bJjugdsoSMVkNRD0rw6NC0POz8RaF7LOAWSPpM0U9JJflx061rarJCUX0X5XZJ+XUO7HpKmhJ3X5fmiQtJISf0T1X9DJmPFGaD9s2PJlnjy0xV1aj95QRlTF2/kF6f3oU1zl9vG8S0GStouaTvwNtDbv97hl0VyGjDXzAaZ2VQzO9vMttavyQlhJODEOQFktDh3HHYCZw7oyPNfrGL3/tjyn+8/UMkfJ86nV0FzrjjOzZoznBxJT0uaL2mcpGaShkj6QNJ0SW/7O3wj6UZJXwFfAm8AR+LtQbgOWAacbWZ54Z1LKgL+ApwvaZakpuGzYkmXS/rcv/eIpO/4zyTdIWmRpI+Aw8LKb5T0laQ5kp6L9vn8tqf5M/m5kv4jqXEt5feGjfW/ko4HzgPu823vVeefgOO7mFnKHEOGDLFAAft8+SbrfusE+8fkxTE1Hf3BUut+6wSbvGB9sDY5DgKUWJL/zwE9AANO8K//A9wCfAIU+GWXAP/xz0uBxv55a//zLuDXtYxzFfCPsOsVQD7QD3gdyPXL/wVcEVFnCN6Lxy7BiJ4AACAASURBVGZ4O60sCY1XjT09gCk1PN+vgSbAKqCPXz4WuKmG8nbAQkARY40BLkz2zzETj4yeOQMUd2/D8MM78uB7i5m/tqpvm99l4859PDRpMUMPK+DUw9on2EJHCrDKzD72z5/C28lkAPCupFnAb4FQGsI5wNOSLieY7ahOwxPfL/yxTgN6RtQ5CXjFzHab2Xa83VZCRGNP5POdiDf7Xm5mi/zyJ/B2Kq+ufBveri+PS7oAqNuLHEfUZLY4FxYiiXu+N4C8prnc/Pws9h2oqLHJ3vIKbnx2JnvKK/jtOc6V1kCIzC27A/jSzIr84wgz+y//3jnAP4HBeIIa71oBAU+EjXWYmd0VQ/to7Il8vphz6ZrZAeBoYBwwAngr1j4csZHZ4lxaCkC7Fo2594IjWLBuBw++t7ja6hWVxk3PzeKTpZu476IjXQ6NFEZSW0nvSlrsf35nM0dJRZKmSfrS95NeEnbvNEkz8PzG3SRd6N/6E3A8MFjSF5K6S8qVdLikLKCrmb0P3Aq0AlrgiXldczhPAi6U1D7suSJfcnwIjPR91S2Bc/261dkTSTdJx/nnPwQ+wnNR9JDU2y//EfBBdeWSWgCtzOwNvBWQA/378Ty7owYyW5zvuuvg6en9O3BJcVf+/cFS3vvquxnrzIzfvjqXt75cx+9G9Od7g1wy/RTnNmCSmR2KJ3C3VVFnN57/9nBgOPBAWPjaw8BleDtorwf+LGk+sB8v+uAYoDUwG5iFJ9jZwFOS5gIzgYfMi7h4Hfie/1IspnhnM/sKz23yjqQ5wLtAYUSdGcDzvi1v8k3Wu+rsiWQhcIP/fG2Ah81sL3A18KLfvhL4d3XleAI8wbfxI+CXft/PAbf4LxDdC8EACTn3U4Li4mIrKSkJrkPpW7uh7Nhbzsh/fszSDbs4pU8Bt53Vl85tmjJ5fhmvzlrDlIUbuOHUXtxyZt/gbHBUi6TpZlZcx7YLgaFmttaPpJhiZofV0mY23surxX77K8zsM0m3Ay3N7L8j6g/Ce4l3Ql1sTBaSegBjzGxoci1xxENG5taojpZNcpl440mMnbaCf76/lLMfmkpOliivMDrkNeaXZ/Th/w3rXWs/jpSgg5mt9c/XATVuTSPpaKAR3rZSANcCb0jaA2wHjq2i2TV4M1WHo95JujhLGgWMAujWLfFJ7JvkZjPq5F5cUtyN/3y8nD3lFZx5eEcGdW1NVpZLalTP5EsK/6o02sxGhy4kvQd0rKLdHeEXZmaSqv0K6M+snwSuNLNKv/hmvJjkzyTdAtyPJ9ihNpfj5c44JZYHknQHcFFE8Ytmdk8s/cTJVrwQN0cak9lujenTYciQ4PpzBEp9uDUk5QFTgD+Z2Ti/rAD41Mx6+dfd8Py55UAZXlzv34FTzKysLvZVYUdXvJjhDnjREqPN7MEg+g4bowney8PGeBOvcWZ2Z5Bj+ONkAyXAGjMbEXT/Do/MfiHoyGTGA1f651cCr0VWkNQIeAUYGxJmny1AK0l9/Osz8OKFh+MtwngEOC8oYfY5APzKzPrjuVBuUPA5KfYBw8xsIFAEDJdUlbsmXn4BzE9Av44wMluci+s0KXOkB/cCZ0haDJzuXyOpWNJjfp2L8RZQXOVHUsySVOTH7F4HvOS/JPwRXoTCZjw3Sgu8aIVZksYTAGa21o+6wMx24Ilb5yD6DhvDzGynf5nrH4F+NZbUBS+2+rHa6jriI+k+Z4ejLpjZJrzVdJHlJfi+YzN7Cm9FXFXtX8GbVR/Ej3JYYWYDAjb3W/jjDAI+S0Df2cB0vE0A/mlmQY/xAPAbXGxzwsnsmbPDkWL4izleAm7yl2IHiplVmFkR3nLzoyUF9odG0gigzMymB9Wno3pSauY8ffr0jZJWBtqp21YqlWlQ6f4k5eIJ89NmFtzuw1VgZlslvY/nRw8qn/MJwHmSzsbzzedJesrMLg+of0cYKRWt4XAkE9/dMCERbg1JwksitNnMbgq6f3+MAqDcF+amwDvAn81sQgLGGoqXGc9FayQI59ZwOABJzwLTgMMkrZZ0TcBDnID34nFY2MvJswMeoxB4319i/QXwbiKE2VE/uJmzw+FwpCBu5uxwOBwpiBNnh8PhSEGcODscDkcK4sTZ4XA4UhAnzg6Hw5GCOHF2OMLwU9im9RiZ8AwOJ84ORyT1ITqJHiMTnqHB48TZ4XA4UpCUWoSi3BzjQAVZzZuQ26FtQsYoX7+Zyl17EzpGTFRWsm/FuoOXjXt0hCzvb2aT3PKEDWuVxvbFGw9e5x2ajwLYCSYvZ2+N9ysrjTXzdx68NrOoB83Pz7cePXrU2bYqKS2FTp0OXm7YsIGCgoJgx4gg0WNkwjOAl2vHzBI7SAqTUomPOFABQOWuvXS562qymjYOtPvKPftYesU9CR2jLqy9/wV2TptHi+MGUPjLiwHo1/G7O4QHTcmdb1M6eQmdhvWm+O4z4+5vWP6CqOqNuXkus94qAy/pfdT06NGDQHfKARg1CkaPrr2eo94JPAlampFSM+fsFk2tctfeb4lU0FQlhKlA5Z59B/9Q1Icwhziwez85zRoF0le04gywb9cBbi3+IKZtqgLfxsyR0sSzjVkmkFIz59wObRM+my385cVU7jk/JWbM4SRDmIGkCDNA4+Yp8l9vyBBvr0mHI8VIuReC9SGaqSbMIepbmB3AjBnJtsDhqJKUE2dH+hHrrNnhcNSOE+cUwc2ak0RhYbItcDiqxIlzCpDOwpz2s+bS0mRb4HBUiRPnJJPOwpwR3HVXsi1wOKrEibOjzqT9rBng7ruTbYHDUSVOnJOImzU7HI7qcOLsqBMZMWt2OFIYJ85Jws2aUwS34tCRojhxTgJOmA/i/v85HNXgfjkcMROES2PMzXMBBsXdUbwUN9jUDY4Ux4lzPeNmzV7SIz8rncOR0UjqIOlxSW/61/0lXRNNWyfOjpgIYtbcuHkORcPbB2CNw5HyjAHeBkJJwxcBN0XT0ImzIylc9bcjAGYm2w7uvDPZFjgym3wzewGoBDCzA0BFNA1TJG9jahKeYzkIkuXSiJztTt7YN5B+AqAy6A5jxq0QdCSWXZLaAQYg6VhgWzQNnThXQ6om5a+NaAS0pjp1Fe60pVMnl1/DkUh+CYwHekn6GCgALoymoRPnKqjcs4+d0+YBsHPavECS8yd61hzUrDa8n3ChzthFJ2vXJtsCRwKQ1BZ4HugBrAAuNrMtEXWKgIeBPDxXwz1m9rx/byrQ0q/aHvjczEb694YCDwC5wEYzO6U6O8xshqRTgMMAAQvNLKrNQaP2OUs6JJqyatpmS5opaUK04yWTrKaNaXHcAABaHDcgZZPzgyeaiRLOUN8JFGb3zsORKG4DJpnZocAk/zqS3cAVZnY4MBx4QFJrADM7ycyKzKwImAa8DODf/xdwnt/uopqMkHQD0MLMvjSzeUALST+L5gFimTm/BAyOKBsHDImi7S+A+Xh/odKCILezSsSsORkz2SBn0ikT5zw48r+0I0M4Hxjqnz8BTAFuDa9gZovCzkslleG5HbaGyiXlAcOAq/2iHwIvm9nXfrvaYkKvM7N/ho2zRdJ1eAJfI7WKs6S+wOFAK0kXhN3KA5pE0b4LcA5wD57/JW1I1RlzKrgY4hHqlIpzdvsHZiodzCzks1oHdKipsqSjgUbA0ohbI/Fm4Nv96z5ArqQpeG6PB81sbA1dZ0uS+TtpS8r2x6mVaGbOhwEjgNbAuWHlO4Dromj/APAbvvHfOOIgWcJc04vC0L1obQvFOUcj0JJGAaMAunXrFlX/MTFqFIweHXy/jiDIlxSe/GS0mR38YUl6D+hYRbs7wi/MzCRZdYNIKgSeBK40s8gIoh8Aj4Vd5+B5C04DmgLTJH0aPguP4C3geUmP+NfX+2W1Il/Qa68oHWdm06Kq/E2bEcDZZvYz34n+azMbEVHn4C9fTn6rId3v/3nKzljrQpAujVQU5uqIxtab+k2aaWZR+xWKi4utJOhERRJE+TvgqF8kTTezOq2vl7QQGGpma33xnWJmh1VRLw/P5fEnMxsXcS8fWAh0NrO9ftltQFMzu9O/fhx4y8xerMaOLDxBPs0vehd4zMxqjXWO5YXM9yTlScqVNEnSBkmX19LmBOA8SSuA54Bhkp4Kr2Bmo82s2MyKK/fsY+kV97D2/hdiMKthkAqujFiYvLHvwaMGkh/n7MhUxgNX+udXAq9FVpDUCHgFGBspzD4XAhNCwuzzGnCipBxJzYBj8N6nVYmZVZrZw2Z2oX88Eo0wQ2zi/F++32UEXmhKb+CWmhqY2e1m1sXMegCXApPNrFpBr9zl/Rt44Wv7YjAts0k3YY4kXKgbXBy1I1ncC5whaTFwun+NpGJJITfFxcDJwFWSZvlHUVgflwLPhndqZvPx3BJzgM/xZsHzqjNC0gmS3pW0SNIyScslLYvmAWKJ1sj1P88BXjSzbZJiaF47Wc2bULlrb8qHr0VLJiQ5SoSYftPnpMD7jpk1a5JtgSMBmNkmvnElhJeXANf6508BT0XWCas7tJry+4D7ojTlceBmYDpRLtsOEYs4vy5pAbAH+KmkAmBvLW0OYmZT8Hw71ZLboS1d7ro6I4Q5KNJ91pzyTJ/urRJ0OBLDNjN7sy4NoxZnM7tN0l/8wSok7cKLJQyUTBHmIGbNDUCYk78I5bzz3AtBRyJ5X9J9eItYDvpqzWxGbQ2jFmdJucDlwMm+O+MD4N8xm+pwACV3vg2psAjF4Ugsx/if4VEnhrewpUZicWs8jOd3Dq1s+ZFfdm0MfTiiJJNnzQd276d08pJkm+FwJBwzO7WubWP5WnmUmV1pZpP942rgqLoOnMlkwotASFyGupxmjeg0rHdC+o6ZRx6pvY7DUUfqayeUCkm9wgbtSYxvHx3Rkcmz5hDFd58JqZBsf9SoZFvgyGzGUMedUGJxa9yC59xehpf6rjvfJANp0IQn5c+UWXM9kfxFKG6FoCOx5JvZC5JuB28nFEnB7oRiZpMkHYqXawO8vKQNfqVIuibldzgc9ULid0LxozWux1tRAzBF0iPRJo7ORCKT8h+adxxRJpyqlnhdGvt2HaBx87TZQyH5oXQOR2Kpl51QXLRGBKGk/KGZc06z+IQ5XsbcPJdZb5VRNLx9aAPVOpPoZdYpE0o3YkTtdRyOOhLPTiixiPNRZjYw7HqypNkxtM9IQkn5Dz9ka+2VayGeWXN4juRZb5Wx74+pO4NOqVC6119PtgWODCQi9304fSRhZi/X1oeL1giAIIQ5XkI5kgGKhrdPWWEGL5QuZGvSOffc2us4HLFzrn9cg5df4zL/eAz4cTQduGiNDOKqvx2RsjPm73wr+NsR3PTWpOSH0k1Ii20tHWmGvw4ESe8A/UO7svi5pcdE04eL1oiToELngoptDkKYg/A3R/k8yQ+lczgSS9ew7bIA1gNRbekTS7RGE+BnwIl4YSFTJf07IhF1g8LFNH+XhrCAxuGIgUmS3uabvNCXAO9F0zCWadZYvH0D/+5f/xBv360atwbPVJwwf5c6CHPyQ+ncAhRHAjGzn/svB0/yi0ab2SvRtI1FnAeYWf+w6/clfVVTA3+2/SHQ2B9rXGjvrXTGCfO3qctseczNcyEVQulGj3ZLuB0JxY/MqDU6I5JYZi4z/NUtAEg6Bqhtt819wDA/BK8IGB7eRzqSCGFOZ1dAXWwPD/tLOtdfn2wLHBmMpAskLZa0TdJ2STskbY+mba0zZ0lz8XzMucAnkr72r7sDNf5mmre1907/Mtc/0vJ7pJstf5e6/lEJhf2ljEA7HInjL8C5/t6DMRGNWyOuJVSSsvH2z+oN/NPMPoumXXgyoWTS0EQ52kiNeGf7V6VKKJ3DkVjW10WYIQpxNrOVdek4rH0FUCSpNfCKpAHhu9VKGgWMAsjJbwUkP5lQQxPkWAnQDZP8ULrx45NtgSOzKZH0PPAq396mqlYfdDRujfeJzhUxxszGVnfTzLb6fQ0H5oWVjwZGAzTp1dkikwlV7jk/oTPoZAtxOvubM4IhQ5JtgSOzyQN2A/8VVmZE8YIwGrfGVVEa8Z01zP4O3eW+MDcFzgD+XFMnkcmEghLmZItwppBxf0w6d3bhdI6EEVopWBfq5NaQlAW0MLPa3joWAk/4fucs4AUzq3W9bCiZUBDC7EQ5OJItzPsPJN8L4kgPJLUFngd6ACuAi81sS0SdIrzMmnl4eYLuMbPn/XtTgZZ+1fbA52Y2UlIr4Cm8VX45wP+a2f/VYEcff4wOZjZA0pHAeWb2x9qeIepQOknPSMqT1BzPLfGVpFtqamNmc8xskJkdaWYDzOz30Y4XrzD367jeCXOABCHM+3YdiKv94rKd7C13ubYcUXEbMMnMDgUm+deR7AauMLPD8dytD/jvxjCzk8ysyMyKgGl844a4AfjKDw8eCvxVUk25gh8FbgfK/X7nAJdG8wCxxDn392fKI4E3gUPwcjqnFE6UU5MxN8/l1uIPQotP6kSlGVMWBhx+d911wfbnSBXOB57wz5/A061vYWaLzGyxf14KlOElwz+IpDxgGN4LPfD8xS0lCWgBbAZqmnU0M7PPI8qimqXEIs65/m4oI4HxfsLolHLWOVGOn6pmyEHszvKtXNMxzKAljZJUIqkkC2PCnLW1N4qF0aOD7c8RJPmhn71/xLKUs0NYwqF1QIeaKks6Gm8bo6URt0bizcBDLtx/AP2AUmAu8Aszq8nfttFPtRzapupCIKr/xLEs334Ez3czG/hQUncgqpUu9UE6CnOyfbhVESmcQdgYvugk1lzT4dE8hb0Pt0nzy9izv4KmjbLjtgvwojWmTw+mL0fQbDSz4upuSnoP6FjFrTvCL8zMJFU7kfTTeD4JXFmF0P4ALwdziDOBWXiz6V7Au5Km1vD+7Qa8/799Ja0BluPlda6VWFKGPgQ8FFa0UtKp0bZ3pD6hba46DVvDb/7ZPdC+g8g13appLnvKK5i8oIxzjiwMxrAZM4Lpx1HvmNnp1d2TtF5SoZmt9cW3Sn+Y77aYCNxhZp9G3MsHjga+F1Z8NXCvv/p5iaTlQF8g0nURsnEZcLr/ri7LzHZE+3zRxDn/spYq90c7WKJIx1lzqhHueiidvIR9uzoHnrQ/3v5aNM6hRYvGTJxbGpw4OzKV8cCVwL3+52uRFfwXea8AY81sXBV9XAhMiEiL/DVwGl7K5A54+e2XVWeEv/P2nfipliV9BPzezDbV9gDR+Jxb+kcx8FOgs3/8BBgcRfuE4oQ5GNJlm6uzj+jI5AVl7NoXX+THQQqdyEfDvgMVbN61n1Wbd7N0w0427NiX6qGN9wJnSFoMnO5fI6lYUshNcTFwMnCVpFn+URTWx6V8k4c5xB+A4/2cQ5OAW81sYw12PAdsAL6PJ/Yb8EL8aiWaOOe7ASR9CAwOTcsl3YX3dcCRIaTyNlchzjmikLHTVjJ5QRnnDuwUf4elpfH3kWGs2bqHjxdvZO6abSwu28GSsp1s3Lm/yrotGufQu30L+hXm0b9THsf1bEevguZ4wQzJw5+ZnlZFeQlwrX/+FF7McnV9DK2irJRvr/arjUIz+0PY9R8lXRJNw1h+CzsA4T+h/dTyBjTRuFlz8KSyMAMU92hL+5aNmTCnNBhxvusu72jAmBkzV21l/KxSPli0geUbdwHQsnEOvTu0YFjf9nRr24wWjXNo1jiHRtlZ7Nhbztbd5WzcuY+F63cwcU4pz37+NQBd2zbl1MPac9aAQo45pC1ZWckV6iTzjqRLgRf86wuBt6NpGOtOKJ9LCmXxH0mUGxU6HEGRnSXOPqKQZz7/mh17y2nZJDe+Du++u8GK8/rte3nms695ddYaVm7aTaOcLE7o1Y7Lj+3Oib3z6dOhRdQzYDNj9ZY9fLBoA1MWlvFiyWrGTltJ17ZNuXBwVy4q7kKn1k0T/EQpyXXATXgzdAOygV2SrscLJMmrrmEs0Rr3SHqTb7ZbudrMXMrHOJi8sW9KhtOlOiOOLGTMJyuYNL+MkYM6J9uctGPm11v4v49X8MbctVSYcXyvdtxwam+GD+hIXh3/2Emia9tmXH5sdy4/tjt79lfw9pfreHH6Kv723iIemryY4QM6cs2JhzC4W5uAnyh1MbOWtdeqmmiiNdqGXa7wj4P3zGxzXQePB+fSaLgM7taGwlZNmDCn1IlzDHy2bBMPTlrMJ0s30bJxDlcc14Mrj+9O93bNAx+raaNsRg7qzMhBnVm1eTdPfbqSZz7/molz1jKkext+NrQXw/q2T7pvOtH4KwkvAw4xsz9I6ornh64y9C6caGbO0/Gm4+H/iqFrA3rGbrLDUXeyssQ5RxTyxLQVbNtdTqtmcbg2SmrbaS39mfn1Fu57eyGfLN1EfovG/Pacflx6dDdaNK6f9wtd2zbj9rP7ceNph/JiySoe+2g51zxRQt+OLfnZqb0554hCsjPXL/0vvLzlw/AiPXYC/wSOqq1hNNEah8RrXdRURheak0mzZufaqBsjBnbisY+W8/ZX67i4uGuyzUlJVm3ezZ/fWsCEOWvJb9GY343ozw+P7hbc6soYad44h6tOOITLju3O67NL+deUpdz47Ez+9u4ifja0FyMHdSY3O/kbsgfMMWY2WNJMADPbUkuipIPEkpVOki6X9Dv/upu/Hj0w9q1Yx9r7X6i9YoYR7dZQ9cHkjX0PHqnMwC6t6Nq2afy5NoqrXR2ctuwtr+Cv7yzktPs/4L3567lxWG8+uGUo15x4SNKEOZzc7CwuGNyFd246mYcvG0yzRtncMm4OQ++bwhOfrGDP/ozKPFjup0wO5dYoIModgGL5XhM5Pd8BvEQU0/NYqG33k0yaNYcTEsNkzaKrEuPIslSa4UtixJGdGP3hMjbv2k/b5lFNRjKeDxdt4HevzWPlpt2MLOrErWf1pbBVakZJZGWJs44oZPiAjry/sIx/TF7CneO/5MFJi7nq+B7JNi8oHsJbhdhe0j14oXS/jaZhLOJc5+l5LAS5+0k6Ut8iHcsMObxuXe379niT6tRHiBFHFvLwlKW8OW8tlx0TbC6QdGPbnnLuHv8lL89cQ8/85jxz7TEc3zs/2WZFhSSG9e3AsL4d+GLFZh6espT7312UbLMCwcyeljQdb0GMgJHRbvgaizjHPD3330yOxVusYsBoM3uwuvqNe3Ss9w1doxGZZHzFr2rMIAQ7qGdJBbdH/8I8ehY0Z8LsOMT5zjuDNSoJfLJ0I79+YTbrd+zjxmG9+dmpvWmSm3z3RV04qkdbjrqqLQvX7aBvjRvapTYRUW5lhC0DjzbKLRZxrsv0/ADwKzObIaklMF3Su2b2VZW1s2p2gQfl0ohV5CLrJ0uYqhu3uudJBQFNJCHXxj8mL6Zsx17at2wSeydpvAClvKKS+95eyOgPl3FIfnNe+unxFHVtnWyzAuGwjnUOD04VwqPcugFb/PPWeMmTag20iGURSszTcz/Z9Vr/fIek+XhJk6oW5wQTlKsgvJ9UEMBUsCFZnDewkIcmLWbinLVcfUIdAos6dUrL/BplO/by86dn8vmKzVx2TDfuOKcfzRql9tL7hkQoyk3So8ArZvaGf30WVezKUhW1Rmv4+U5D0/TQ9PwZYH3E1L22fnoAg4DPom0TTjyz5mH5CxLmw02ll2QNkd7tW9K/MI9XZ9VRYNcGvLNKPVCyYjMjHvqIOWu28sAlRdzzvSOcMKcux4aEGcDM3gSOj6ZhNKF0z/if04GSsCN0XSuSWuBFdtwUuWNA+DZEFdt3RdNdTNSHeDqBTi7fG9SZ2au2HkzYk8m8OnMNP3j0U5o2yuaVn53gVkimPqWSfiuph3/cgbfFVa1EI873+p/9zKxn2HGImdW6OtDfd/Al4GkzeznyvpmNNrNiMyvOzgt2GWl9imY6CPSB3VWnfUx3zh3YCckTrpgZnPSU5FFhZvxryhJuen4WQ7q3YfwNJ9KvsNqcOY7U4Qd4m8a+greDd4FfVivRiHMouuKTWK3y15U/Dsw3s3rdMSVosYxmU9JUFuiSO9/mjTMfpeTOqLIVphUdWzXhuJ7teHXWGrzdg2IgDfYPrKg07hz/JX95ayHnDuzEEz8+Or4l6456w8w2m9kvzGyQmQ02s5uizUcUjTiXSxoNdJH0UORRS9sTgB8Bw8J2Gjg7GsPCidXfHLRIjrl5LrcWf8CYm+cG2m99cWD3fkonLwG8LagycQY9clBnVm7azaxVW2NrOCqWDZ3rn4pK4zfj5jB22kpGndyTBy8ponFOeobJOWIjGnEeAUwG9uD5mSOPajGzj8xMZnakmRX5xxs1tUk1wvfWm/VWWVQz6FQjp1kjOg3rDUCnYb3JaRbd2qF0EvHhAzrSKCeL12J9Mfjoo4kxKABCwvzSjNXcdPqh/PfZ/Rp64voGRTSJjzYCz0mab2azq6sn6XYz+59ArasDQc+aQ3vrzXqrLKX31quN4rvP5MCtp0YtzCV3vk3p5CV0Gtab4rvPTJRZgWW5yWuSyxn9OvD67FLuOKdf2ifQiRTmm07vk2yTHPVMLHHO1Qqzz0VAUsU5UT7fdNhbLxpimTF/yw0Sg6hHi+/7HhRkn+cXdWLi3LV8tHgjp/ZtH2TX9YqZ8f+9Ns8Jcxoj6e/4q6mrwsxurK2PINUmo79vpbswx0LIDRKaOQcpzPPXdaByz76D4h8kQw9rT5tmuYybsTp6cV5ThwiPBPOPyUt4+rOvuf7knk6Y05e4E4UHqTgxviaPjmhfBqZypEQ6EqsbpDrmr/vuHsBZTRvT4rgB7Jw2L66+I2mUk8XIQZ156tOV0Weqmz7dWyWYIjz3+df89d1FXDCoM7cOb7grP9MdM3si3j6CdMzFP3OuItl+Vb/cjvqhLsI8f12Hbx3V4Se4CnwPykuO6kp5hfHyjNXRNTjvvKBNqDPvLyjjjlfncXKfAv584ZHu5V8cSGor6V1Ji/3P72xcKKlI0jRJX0qaI+mSsHunSZrhR5h9JKm3MRmXjAAAFrpJREFUX95Y0vOSlkj6zF/5XJMdBZL+V9IbkiaHjmieIZZk+yfUUvZitH1VR0NNtp+uRApxHf6QRrf1TQz07ZhHUdfWvFCyKvaY5ySyaP0Ofv7MDPoVtuThywan/QvNFOA2YJKZHYqXm/a2KursBq4ws8OB4cADkkKZox4GLjOzIrxV0qEkb9cAW8ysN/A3oLbceU8D8/ESHd2NtwfrF9E8QCxujb8DkcupDpaZ2Z9i6Ktaaku235CJVfyCyuKXbt9eLjmqK7e/PJeZq7amxU7PW3bt59onSmjWOIdHryimeT3t7ZfhnA8M9c+fAKYAt4ZXMLNFYeelksrwVvBtxXPThpZgtuKbJdfnA3f55+OAf0iSVT8TaGdmj0v6hZl9AHwgKRhxlnQcXqKOAkm/DLuVBwQeDV+XZPup4m8OMjtcEIKYbqIaFOcO7MQfJnzF85+vql2cH3mkfoyqhvKKSn769HTWbd/Lc6OOTdldS9KQDn5WTIB1eDnlq8Xfcq8RsNQvuhZ4Q9IeYDtwrF/eGVgFYGYHJG0D2gEbq+m63P9cK+kcPJGPKmFcNH+iGwEt/LrhSVa34+V0DoxkJNtPJRqqmNaEpFHAKIBu3bpF1aZF4xxGHFnI63NK+d25/WveZTrJKwR///pXfLpsM/dfPDAtZvn1TL6k8KiH0WY2OnQh6T2gYxXt7gi/MDOTVK2PS1Ih8CRwpZmFXG03A2eb2WeSbgHuxxPsWPmjpFbAr/A8DXl+37USzSKU0FR8jJmtBJCUBbSIzDAXN7Uk208EZ7f4ijd29q/3cSNxwlw1/i/jaIDi4uKonciXHNWVF0pWM3FOKZccVYOoS5Ak3/RL01fz5Kcrue6kQ7hgcJek2JDibDSzanfgNbPTq7snab2kQjNb64tvWTX18oCJwB1m9qlfVgAMNLNQeuPngbf88zVAV2C1pBw8l8emGmyc4J9uA06trl5VxOLc+h9JPwEq8BzaeZIeNLP7YhkwVTi7xVffOU+GSDdEUT5Q2izhYwzu1oZD27dg7LSVXFzcFS8HV+rwZek2/vuVuRzbs60LmUsM44Er8bJqXgm8FlnB3wP1FWCsmY0Lu7UF/v/2zjxKquraw9+vm6YbaBAZZLIBUYanMoitQgKoJPqiMeoyikkQVMQhKyYvwWgm86J5MRJNTIyJYzSIxFmTOGBiEAfihMxIEBVQROZB5qHp3u+PewuLtqbbdav6VvX51rqrT5060+3u+vXufc/Zm4Mk9fH90qfgPdSLH/d1PM/B9ET+ZknXmNlNyQ6jZHIIJYipeqRvKZ8NPIf39HFMgP4NJmwBixfmTOpzRVMR5n2rWh5w5QNJXPz5w1i0aitvLs8oCFje2LKzhm9OmUPblmXc9vXBNHM7M3LBROAUSe8BX/RfI6la0p/8NqOAEcBFcYHZBpnZPuBS4AlJ8/F07mq/z71Ae0nvAxNIvAsEPhXzWOz7jGMSxQhiOZf5sZnPBv5gZjWp/DhRJZ0AN9TNEfRhYDELc74EOB3nDO7Gr59fwp9mLGNIr/aJG51xRl7XVFdnTHh0Hqu37OLhy4bSsbXblZQLzGwjXkq9+vWz8H3HZjYFmJKk/1/xrOr69bvxQlWkm/9pv7jTzA7YZiwpbX8IZjnfhbdHrxXwiqQeeA8FHQEpJmGubxVHRZgBKspKuWBID6YtXsey9dsTN3r66cT1OeKOl5fywjvruPbLR3JsD/cAsAnwowzrPkPG4mxmvzezbmZ2unl8SEAHdzakErRCSnBaiMKcSICjJsTJGDOkB81LS7jv1eWJG3zlK3lby2tLN/Cb572A+WOH9sjbvI78I+k039/crV4M/ElARnGHM3ZrSOoE/BLoamanSToSGIrng8kL+3buDT06Whhk+schW2EuBDGMGh1bl3P2MV15fPZKrjqlLwfXj7fxzDOJO4bM2q27+c5Dc+nVsZKJ5/SP3ANKR+iswvM3n8mBPuZtZLiVLohbYxLwTyAWJeZd4LupOki6T9I6SRlFuLGa5MtZfcujRZtmKRWFZKVGlUuG9WJ3TR0PzlzRKPPX1NZx5YNz2Lm3ljtGD3YnAJsAfojlKcCrZnZ/3PWkmW3OZIwg4tzBzB7Fj4fgP9GsTdNnEt6Z9YxJJEJ1u/bsj2BWqGmWgljNTUGQW67M3w6Fvp1bM6JPR+7793K27a5J3yFkbnh2MW99sJkbz+lP706t03dwFAVmVgtU+Vv2AhPkE7JDUnv8PXuShuBtrE61uFeArPcxxUJMQrA0SxWbcy/imbg0ggpzsdFyZclnrnxz1Sl92LhjL3e/suzAN3J8AOWvc1cy6bUPGPf5wzhrULeczuWIJMuBVyX9VNKE2JVJxyCfkgl4G7B7SXoVmAx8O/ha05NIoLpMGMXhk3+SNGVSIpE86YZ3E7RMTWOdFiwWSzkKQpyIgVVtOWNAF+6ZsYy1W3d/+sbddyfvlCWLVm3hR08u5ITD2vGj0wvnobUjVJYCz+Bpbeu4Ky1BPjn/wdv39xawFrgHz++cFZIukzRL0qza7TtSti1pUZ6xFdp2+U76PruWth/sPKA+3+KbyXoLUZQTiXBUhDgZV/93X2rrjN9Ni/u1vfzynMy1acdeLn9gNm1bNOePLgRok8XMrk90ZdI3yJOJyXj7mmOhQb+BFywkow3VyYiPnVDevWr//5j7VrWkWdedSfvVR3VGWVxm7D7PeeEy+0xdy9wLq/bX17QoZer2IxMeRgkq3OlcGo0hzEEEcuehB4ZTjrq4ZkuP9q24YEgP7vfdDLny/+6uqeXSybNYt20Pj14+lA6V7qBJU8WP03ENcBRQEas3s5Hp+gYR56PNLF69XpSU3/POPovXdPpMrGKZcfAfd3L+5FmU1H7qRxx62zKG3raMulIx85s9mXnFYRiN576Ip6HCHJaIFrsYJ+LbI3vz+KyV3PjcO9x7YXXoiS/r6oyrHpvP7A83c/vowQyqapu+k6OY+Qte4KQzgCvw4nKsz6RjkE/nHP8hIACSTiBNEkNJD+EFCOkraaWkSwLMF0i86kpLeHjc8Tzx52PY2ulAS2Vrp3KemHQMb36rF1YazscxW6s5qDAXiusg6rRr1ZwrRx7B9HfW8cScj+Gpp0Id/+bnl/DsgtX8+PR+nN6/S6hjOwqS9mZ2L1BjZi+b2TggrdUMmQXbX4i3Q6MMeE3SCv91DyBllHsz+3omiwiTKYcNpfOYrQz/9afZneePrWJVdXSOygYR5mIT48qPQ89MFZjxw3vx4pJ1/PRvb1M9qh89Qxr3TzOWccdLSxl9QncuHd4rpFEdBU5Og+3nNzJMPZL5nhO5NmIcPm09NRUlvP3lTvR/Zi2HT1vPnHHhHZcNw9ecjkIX5SiIcDJKS8StXzuG026dQc8Bfdi9dx8VZdkl9bn9pfe56R9LOL1/Z64/8yh3AtARI6fB9j/Mbm3ZE+ThYLv129m7tRmPPHIcG/tUsmhsd7509SJardvDjkMa/8FMJlZzIQlzlEU4FZ3aVPDr8wbA/8Ivpy7m52cd3eCxbp32Hr+d9i5nDerKb84b6EKAOpBUgedjPgIvtdW9ZpazYPuRI5H1XFpbx9V3n8uwbp5bY2OfSh5+pJqWm8I5GZaN1VyIwpyp+NbW7KG0rPH/+AVhZD/vZzX59Q9pU1HGVaf2CWTx7q6p5YZnF/PAGx/y1cGHctO5AygtcRazA/CSytYAM4DTgCOB/wkyQKTEuWSvJ071t3hB5tbz+s5ewtzpG/rtT/xaW1HKtq7Z56LNpTsjW1FuTAv2vX8/wKYV82nXfSC9h+Ul/0Jo1I0fz9eOq+IPL77PR5t3ctO5Ayhvlv535b212/j2Q3N5Z802LhvRix9+qR8lTpgdn3KkmfUHkHQvMDPoAJES54aQyvccL9DZkm1Y0rD3M0fFnVBbs4dNK+YDsGnFfGprRhWUBV1yzz3caEZVu5bc/M8lrN6ym1+cfTR9kuyB3rRjLw/NXMFt09+jVfNm/Pni4zi57yF5XrWjANj/r7qfpTvwAJEU56DWc74EOhXZuDMytZqjIsjxlJaV0677wP2WcyEJMwDHHotmz+ZbJx/BoQe34JrHF3Dqb1/h2B4Hc/5xVVQd3JLd+2rZuaeWf/1nDVMXrmFvbR0j+x3CxK/255DWFenncDRFBkqKJSMR0MJ/LbyE4G3SDRBJcYbkAt0QshXoXAbzL19WA83TC1oUhTlG72Fj0lrMrZfvyuOKAjBnzv7iWYO6MeyIDjw552MemrmCax5fcEDT1uXN+PrxVYwe0iOpZe1wAJhZ1n7UyIpzMhpiPUPDBTrbqHOprOaNt09h65J5tOk7iKozxyZsE2VRjqe0rDy6AhyA9pXlXDqiF+OHH8bCj7ewY08t5WUlVDQrpWeHlrRsXnAfGUeBEunftDDdGxBMoMPIbpJKmMuX1bB1yTwAti6ZR+3e8ymtZ0FXflwXuV0QxSDAB9Al8Sk+SQw41B29djQekRZnCNe9AQeKbiKhzlc+wtLm5bTpO2i/5ZxImPOxC6LoxDYoq1Y19gocjoREXpyT0VDrOZ5shbihVnPsAWDVmWNTWswN2QXR5MU2KNdd512OokJSO7yAQz2BD4BR9dNDSRoE3IF3aq8WuMHMHvHf+wJwM178oe3ARWb2vh8ofzxektb1wLhcHdSL1omHJCTbzZBMAPOR4TpbYY6RSJjh010QQNpdEK2X79p/OQJyfUahdR2Fxw+BF8ysN/CC/7o+O4GxZnYUXjq930mK+bLuAEab2SDgQeBav34uUG1mA4DHgZtydQORspxLUxziC+reCGJBByVX4l//4V+qXRBOiB2OlJwFnOSX7wdeAn4Q38DM3o0rr5K0DugIfIIX3C223e0gvIBFmNmLcUO8AVwQ/tI9IiXO4AnU9m6ZG/SpTg7mQqCzCQWaaj9zsl0Z9YXZibLDkRGdzGy1X14DpPzgSjoeaI6XVgo818VUSbvwkowMSdDtEuC5cJb7WSInzpBcoBtytDtMgc63MNenUIS52dKPQxtL0mXAZQDdu3cPbdz9zEoZktzRuHSQFP8DutvPnASApGlA5wT9fhL/wsxMUtJMvpK64GV1utDMYh/G7wGnm9mbkq4GbsET7FifC4Bq4MSA95QxkRTnVDSWQOfDj52KqAlzmAKcivg0ZtXV1blNle2IGhvMrDrZm2b2xWTvSVorqYuZrfbFd12Sdm2AZ4GfmNkbfl1HYKCZvek3ewT4R1yfL+L9ATjRzPYEvalMiaw4B3VvpCMmrkFFOlNRzqXV3FjCnC8BblSqq8Gc5hchT+GlhJrof/17/QaSmuMlrZ5sZo/HvbUZOEhSH98vfQqw2O9zDHAX8CUzSyj4YRFJcY4dvAjTvREjU5EOYikXqjA3lvjuq6uhWUlZo8ztaDJMBB71U+N9CIwCkFQNXGFm4/26EUB7SRf5/S4ys3mSLgWekFSHJ9bj/PdvBiqBx/xgRivM7Mxc3IAsQlZDZfsqK69sf8DBi1TWc7LdG0GydmdLNkGN0olzGMIcNet33pZ/sWbPUjqXH86aPUtnp/q3tT7V1dU2K2wfseQs54giKdDvR7ERKXGWtAFoH1c1Fwg7uEQHYEPIYxbjHLkYvwQ4Ju71RjPrkGlnSevxrCBH06CHmXVs7EU0FpES53wgaVau/xoXwxz5uAeHw5Gcgjgh6HA4HE0NJ84Oh8MRQZqiON+dvombIw/jOxyOFDQ5n7PD4XAUAk3RcnY4HI7I48TZkRA1JF2ww+EIDSfOBYKkoySdKKl9+tYNnmOYpDGwP1iME2iHo5GI5PHtXCCp1MxqczT2EUBbYGEuAqFIOg34FbAMKJN0iZmtCXH8EqAlXswASWplZnf6Al0SF6nL4XDkiaK3nCX1ATCzWklZpytPMP4ZwJN4Z+4nxeYLcfyTgFuB8WZ2NrAXODrMOcyszsy24wUlvxf4nKTvxd4Lcy6Hw5EZRS3OvnDOk/QghC/Qkj6HJ8oXmtnJeAFSEqXDyYa1wOVmNlNSZ+AE4EpJd0k6N2TXwz6gCk+kj5d0i6Qb5VHUvysOR9Qo2g+cpFbAlcB3gb2SpkBOLOhfmdlcv/wzoJ2k9JlYM8TMFselxrkEuN23oF8HzsWLgREWfwfWmNkLwCzgCqCNeTgL2uHII0W9z1lSV7wUMxXAncBuMwst55cv8q3MbKtf7gI8DZxqZusltTezjWHNl2D+qcC1ZjYnpPG6AjcArwHX4GWHOA54xszuCmMOh8ORGUVrOYOXtNHMtpvZBuByoEXMgpY0WFK/LMevNbOt/kvhJYbc5AvzaOAXklpkM0eM+u4LSV/Fy4u2Kozxwft+AR8BPwUmmNnP8dLzTA1rDofDkRlFbTnXR1IHPB/xUKAUONnMVoY8xyRgNXAqXuDuhSGPX46X8XcCcL6ZvR3y+FXAIWY223/tdms4HI1Ak9lKB2BmGyQtAE4DTglTmH3LtgwY7n/9gpm9F9b4cdThif85ZrYk7MHN7CPgI0lyvmaHo/FoapbzwcCjwFVmtiBHc1wEvGVmi3IxvsPhaBo0KXEGkFRhZrtzOL6sqX1THQ5H6DQ5cXY4HI5CoKh3azgcDkeh4sTZ4XA4IogTZ4fD4YggTpwdDocjgjhxDhlJN0ta5H+9QtLYNO2vk/T9BPU9JaU8YCLpJUk948rV2aw9xTw9JX0jF2M7HI7ENKlDKDEkNTOzfTka/jKgXa5iRzcSPYFvAA828jocjiZD0VrOksZKWiBpvqQHJE2SdKekN4GbJA2S9Ibf5q/+ARUkfUfSf/z6h/26EyXN86+5klonmfMpoBKYLen8eKtY0uGS/iFptqQZieJ6SDrWX+984Ftx9UdJmunPv0BS7yS3PcZv87ak4/2+7ST9ze/3hqQBaeoT3etEYLhf970G/UAcDkcwzKzoLuAo4F2gg/+6HTAJeAYo9esWACf65Z8Dv/PLq4Byv9zW//o08Hm/XAk0SzH39rjydcD3/fILQG+/fAIwPUGbBcAIv3wz8LZfvg0Y7ZebAy388ktAz7jyPX55RL2+P/PLI4F5aeo/c6/ASXiR6Rr9Z+sudzWVq1gt55HAY+ZFo8PMNvn1j5kXz/kgPOF92a+/H0/QwBPIv0i6AC/4PMCrwC2SvuP3C+QSkVQJfA54TNI8vHRQXeq1aeuP/Ypf9UDc268DP5b0A6CHme1KMtVD/v2+ArTxxxwWG8vMpgPtJbVJUZ/VvTocjnAoVnFOxo4M2nwZ+CMwGHjL909PBMYDLYBXGxBqtAT4xMwGxV3/lWlnM3sQOBPYBUyVNDJZ0zSvM5kr23t1OBwhUKziPB04T36maknt4t80sy3AZknD/aoxwMt+KqYq8zKP/AA4CKiUdLiZLTSzXwFvAYEEy7yYz8slneevR5IG1mvzCfCJpGF+1ejYe5J6AcvM7Pd42UoGJJnqfL/9MGCLf58zYmPJy0e4wV9Pwvok97oNSOhndzgcuaEod2uY2SJJN+AJbi0wN0GzC4E7JbXEy2p9MV6M5ym+20PA783sE0n/J+lkvHCdi4DnGrCs0cAdkq7FCyn6MDC/XpuLgfskGfB8XP0ovId9NcAa4JdJ5tgtaa4//ji/7jp/zAXATv++U9V/N8G91gG1/oPKSWb224D37nA4AuICHxUwkl7CC+j/QSMvxeFwhEyxujUcDoejoClKt0aukdSfA3dTAOwxsxPyvJRJeHkLHQ5HkeHcGg6HwxFBnFvD4XA4IogTZ4fD4YggTpwdDocjgjhxdjgcjgjixNnhcDgiyP8DbIRmA50+o18AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 288x288 with 4 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "_ = plot_objective(metadata_boosts, sample_source='result')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Seems that there's not much to tune here, but let's keep going."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Evaluation with: MRR@100\n",
      "Score: 0.3083\n",
      "CPU times: user 2.26 s, sys: 678 ms, total: 2.94 s\n",
      "Wall time: 4min 7s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "_ = evaluate_mrr100_dev(es, max_concurrent_searches, index, template_id, params=final_params_boosts)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So that's the same as without tuning. What's going on?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Debugging\n",
    "\n",
    "Plot scores from each sub-query to determine why we don't really see an improvement over individual queries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "from itertools import chain\n",
    "\n",
    "from qopt.notebooks import ROOT_DIR\n",
    "from qopt.search import temporary_search_template, search_template\n",
    "from qopt.trec import load_queries_as_tuple_list, load_qrels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "def collect_scores():\n",
    "    \n",
    "    def _search(template_id, query_string, params, doc_id):\n",
    "        res = search_template(es, index, template_id, query={\n",
    "            'id': 0,\n",
    "            'params': {\n",
    "                'query_string': query_string,\n",
    "                **params,\n",
    "            },\n",
    "        })\n",
    "        return [hit['score'] for hit in res['hits'] if hit['id'] == doc_id]\n",
    "\n",
    "    queries = load_queries_as_tuple_list(os.path.join(ROOT_DIR, 'data', 'msmarco-document-sampled-queries.1000.tsv'))\n",
    "    qrels = load_qrels(os.path.join(ROOT_DIR, 'data', 'msmarco', 'document', 'msmarco-doctrain-qrels.tsv'))\n",
    "    template_file = os.path.join(ROOT_DIR, 'config', 'msmarco-document-templates.json')\n",
    "    size = 100\n",
    "    \n",
    "    cross_field_scores = []\n",
    "    best_field_scores = []\n",
    "    \n",
    "    with temporary_search_template(es, template_file, 'cross_fields', size) as cross_fields_template_id:\n",
    "        with temporary_search_template(es, template_file, 'best_fields', size) as best_fields_template_id:\n",
    "            for query in queries:\n",
    "                doc_id = list(qrels[query[0]].keys())[0]\n",
    "                \n",
    "                cfs = _search(cross_fields_template_id, query[1], cross_fields_params, doc_id)\n",
    "                bfs = _search(best_fields_template_id, query[1], best_fields_params, doc_id)\n",
    "\n",
    "                # keep just n scores to make sure the lists are the same length\n",
    "                length = min(len(cfs), len(bfs))\n",
    "                cross_field_scores.append(cfs[:length])\n",
    "                best_field_scores.append(bfs[:length])\n",
    "\n",
    "    return cross_field_scores, best_field_scores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "cfs, bfs = collect_scores()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAD4CAYAAADrRI2NAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dfYxcV5nn8e/jdtmUDZtyksaK2zY2g+Vos1nboUWMjNDEWeJJGHAvhAQEg2Gj9WqHWfGy46WzE5Ewi4RZa4YXaZSRZ8KsM2TB5IWOIeyYbByENlpnpk07cULwxoQkdpHEBtxmiBtot5/9o061q6tvVd3qertV9/eRWl116lbVuan46VvPOec55u6IiEh6zOt0B0REpL0U+EVEUkaBX0QkZRT4RURSRoFfRCRl5ne6AwCXXnqpr1q1qtPdEBHpKocOHfq5u/fX+7xEBP5Vq1YxOjra6W6IiHQVM3thLs9TqkdEJGUU+EVEUkaBX0QkZRT4RURSRoFfRCRlEjGrR0Sk242M5dm1/yg/G59gWS7Lji1rGdow0OluRVLgFxFp0MhYnlsfOMLE5BQA+fEJbn3gCEAig79SPSIiDdq1/+h00C+amJxi1/6jHepRdbriFxFp0M/GJ+pqh86mhnTFLyLSoGW5bF3txdRQfnwC50JqaGQs38JeXqDALyLSoB1b1pLN9M1oy2b62LFlbeTxnU4NKdUjItKgYoombupmLqmhZlLgFxFpgqENA7Fz9MtyWfIRQb5SaqjZlOoREWmzelNDzaYrfhGRNqs3NdRsCvwiIh1QT2qo2ZTqERFJGQV+EZGUUeAXEUkZBX4RkZRR4BcRSRkFfhGRlFHgFxFJGQV+EZGUqRn4zWytmR0u+fmVmX3CzC42s4fN7Nnwe0k43szsK2Z2zMyeNLOrWn8aIiISV83A7+5H3X29u68H3gycBb4FDAOPuPsa4JFwH+B6YE342Q7c2YqOi4jI3NSb6rkW+Im7vwBsBfaE9j3AULi9FbjbCw4COTO7rCm9FRGRhtUb+N8PfD3cXuruL4XbLwNLw+0B4HjJc06EthnMbLuZjZrZ6KlTp+rshoiIzFXswG9mC4B3A/eWP+buDng9b+zuu9190N0H+/v763mqiIg0oJ4r/uuBH7r7K+H+K8UUTvh9MrTngRUlz1se2kREJAHqCfwf4EKaB2AfsC3c3gY8WNL+4TC7ZyNwpiQlJCIiHRarHr+ZLQbeAfyHkuadwDfN7BbgBeCm0P5d4AbgGIUZQB9tWm9FRKRhsQK/u78KXFLW9gsKs3zKj3XgY03pnYiINJ1W7oqIpIwCv4hIyijwi4ikjAK/iEjKKPCLiKRMrFk9IiJxjYzl2bX/KD8bn2BZLsuOLWsZ2jCrakviXjtNFPhFpGlGxvLc+sARJianAMiPT3DrA0cAGg7QrXzttFGqR0SaZtf+o9OBuWhicopd+48m+rXTRoFfRJrmZ+MTdbUn5bXTRoFfRJpmWS5bV3tSXjttFPhFpGl2bFlLNtM3oy2b6WPHlrWJfu200eCuiDRNcZC1FTNvWvnaaWOFmmqdNTg46KOjo53uhohIVzGzQ+4+WO/zdMUvknKaG58+CvwiKaa58emkwV2RFNPc+HRS4BdJMc2NTyelekRSbFkuSz4iyJfPjZ/rOIDGD5JJV/wiKRZnbvxtI0f45N7D5McncC6MA4yM5au+dnH8oN7nSesp8Iuk2NCGAT7/nisZyGUxYCCX5fPvuXL6qvy2kSN87eCLlE/6jjMOoPGD5FKqRyTlhjYMRKZfRsby3HPwxYrPqzUOoPGD5IoV+M0sB/wt8K8AB/4dcBTYC6wCngducvfTZmbAl4EbgLPAR9z9h03vuYg0VXk+/tXfnpt1pV+qVo2cuOMH0n5xUz1fBv7B3S8H1gHPAMPAI+6+Bngk3Ae4HlgTfrYDdza1xyLSdFH5+PGJyYrHG9SskaPaOslVM/Cb2UXA24G7ANz9d+4+DmwF9oTD9gBD4fZW4G4vOAjkzOyypvdcRJomKh9fzQc3rqw5O6fW+IF0TpxUz2rgFPB3ZrYOOAR8HFjq7i+FY14GlobbA8DxkuefCG0vlbRhZtspfCNg5cqVc+2/iASNTJ2sJ+/+oY0r+dzQlbGOrTR+IJ0VJ9UzH7gKuNPdNwCvciGtA4AXKr3VVe3N3Xe7+6C7D/b399fzVBEp0+jUybh59yWLMrGDviRXnMB/Ajjh7o+H+/dR+EPwSjGFE36fDI/ngRUlz18e2kSkRRqdOrljy1oy86zmceNnK+f9pXvUDPzu/jJw3MyKIzLXAj8C9gHbQts24MFwex/wYSvYCJwpSQmJSAs0Zepk7bivGTk9Iu48/v8E3GNmC4DngI9S+KPxTTO7BXgBuCkc+10KUzmPUZjO+dGm9lhEZml06uSu/UeZnKqerdWMnN4RK/C7+2Egqtj/tRHHOvCxBvslInXYsWXtjPLKUF+grvbNwEB1dnqMVu6K9IBGtyWs9I1hyaIMY5+5rql9lc5T4BfpEY1MndyxZS077ntiVrrn1785x8hYXlf6PUZF2kSEoQ0DLF4w+zpw8ryrqFoPUuAXEQDOVCjRoKJqvUeBX0SAyjOANIWz9yjwiwigompposFdkR41l9o9r8nMm54SmstmuOPdV2hgtwcp8Iv0oGLtnmIQL9buASpuulK+DuC35863p7PSdgr8Il1uZCzPHfuenq6fv2RRBncq1u6JCvzVav3oir/3KPCLdLGRsTw77n2CyfMX5t+frlJIrd6aPprR05s0uCvSxXbtPzoj6NdS78wdzejpTQr8Il2snityo5Dr37TzwKw6/ZrRky4K/CJdrNoV+ZJFGQbC48aFnZKiNmnRNonpYoVimp01ODjoo6Ojne6GSNeJyvEDZPqMXTeuY2jDAJt2HogswDaQy/LY8OZ2dVVawMwOuXtU5eSqNLgrkjBRs3Ruf1f0fPpiW7XjNXAr5RT4RRLktpEjfO3gizPaTp+dZMd9TwDRc/BrVeVsdJMW6T0K/CIdVLq6NrcoU3Eq5uSUz3lOfaObtEjvUeAX6ZDy1bLV5t/D3FMzjW7SIr1HgV+kQ6JWy1bTSGqmkU1apPdoOqdIh9RzBZ/pM6VmpGliBX4ze97MjpjZYTMbDW0Xm9nDZvZs+L0ktJuZfcXMjpnZk2Z2VStPQCRJRsbybNp5gNXDD0UulCpVzxX84gXz+eTewzVfcy79kPSpJ9Vzjbv/vOT+MPCIu+80s+Fw/9PA9cCa8HM1cGf4LdLT4lTELB3MvSibIdNns/a5jVKcqlmrymbcfki6NZLq2QrsCbf3AEMl7Xd7wUEgZ2aXNfA+Il2hWoVLuBCQ8+MTOCGYz2H9ZOlrzqUfInEDvwPfM7NDZrY9tC1195fC7ZeBpeH2AHC85LknQptIT6u1UCoqINdTYC3Oe8Xph0jcVM/b3D1vZq8HHjazH5c+6O5uZnX9Hxz+gGwHWLlyZT1PFUmkWgulmhl4q40PaMGW1BLrit/d8+H3SeBbwFuAV4opnPD7ZDg8D6woefry0Fb+mrvdfdDdB/v7++d+BiIJUavCZTMDb7UZPqq0KbXUDPxmttjMXle8DVwHPAXsA7aFw7YBD4bb+4APh9k9G4EzJSkhkZ41tGGA9755gD4zAPrMeO+bL8yfryfwFl8jypJFmaqDtKq0KbXESfUsBb5lhf8R5wP/093/wcz+Cfimmd0CvADcFI7/LnADcAw4C3y06b0WaZK5bEhe7bXuP5RnKlS8nXLn/kN5Bt9w8fQCqtJiapVkM318/j1XAkSWWrj9XVfU7IsWbEk1NQO/uz8HrIto/wVwbUS7Ax9rSu9EWqjZ0x7j7Fv7h+su456DL1adzFN+da5SC9JsKtkgqdXsDcYrDd4Wd73Kj0/M2BAlykAuO+O9deUuraDAL6kVd9pjeX38xQv6yPTN48zEJMtyWa65vJ9Hf3yqYkAvbnkI1YO+BmClXRT4JbXiTHuM2uHq1d9NARfSQ+X188vFmec8oDSOtJGKtElqxZn2uGv/0TkvssplM7GOM+Cx4c0K+tI2CvySWnGmPTay6GrxwnhfqB1URE3aSqkeSbW5bltYS59ZXX805jqgLDIXCvwizJ7PXxywnUvQh8Ic/oE6/miojo60k1I9knrlVTOLA7aVgvbC+bX/2RQHa8vHECpRHR1pJ13xS1eKu+I2znFxt0AcyGV5bHgzABv+/HsV98gtDhBH7XV7zeX93H8or43PpaMU+KXrxF1xG/e4uGmW0uPGq2yMXlqfJ2oMYfANF2s1rnSUAr90nbgrbuMeF3cAd1kuO/0NotoEz0d/fKrq62g1rnSaAr90nbgrbisF89L2kbE8r/72XM33NOCay/tnFU2rp38iSaHAL10n7kYjfWbTlTLLrR5+iIuyGV793blYe946hSv5OGMBGqiVpNOsHuk6tVbcjozl2bTzQMWgD0zveRsn6BfFSQcZ9dXdF+kEXfFL14maLVMcIC0f0G0nAz64caXy95J4CvzSlSoNkMadmtksxTLLKrIm3USBX3pKOwdWFeylWynwS1crX6B1UTZTc2vDZliyKDO9mCuqH/qDIEmmwC9da2Qsz477npgeoM2PT9A3z8jMszmXUo6rdNy42Vs4irSaZvVI1/rst5+eNStn6ryzYP682LXw5+pMybeKagvFRJJIV/zStSrVyindIatVSufqx11QJpIUsa/4zazPzMbM7Dvh/moze9zMjpnZXjNbENoXhvvHwuOrWtN1kdZblJmHlbWVF1WrtGBLC7kkqepJ9XwceKbk/heAL7r7m4DTwC2h/RbgdGj/YjhOpOlanc4BWDC/jw9uXFl1l644WziKJEmswG9my4F3An8b7huwGbgvHLIHGAq3t4b7hMevDceLNNUd776i5YNU4xOT3H8oz44ta/npzndG7o0bZwtHkSSJm+P/EvBfgNeF+5cA4+5erG51Aij+Xz4AHAdw93NmdiYc//PSFzSz7cB2gJUrV861/5JSxemT59vwXlEVPcup4qZ0k5qB38z+EDjp7ofM7Peb9cbuvhvYDTA4ONjauXfSNeLMh+9EWQYN1EoviXPFvwl4t5ndALwG+BfAl4Gcmc0PV/3LgXw4Pg+sAE6Y2XzgIuAXTe+59Jxa8+GLfxTmug9uNYsX9IXZQNE0UCu9pGbgd/dbgVsBwhX/n7r7B83sXuBG4BvANuDB8JR94f7/DY8fcK9SJlFSqfTKPrcogzuRK25L58N/au/hlqV2zlYJ+hqolV7TyDz+TwPfMLPPAWPAXaH9LuDvzewY8Evg/Y11UXrNbSNHuOfgi9O7WFWaj1/0s/EJbn3gyZYF/YFwNR/1TaLPTAO10nPqCvzu/n3g++H2c8BbIo75DfC+JvRNetDIWH5G0I/DDCYmWxP2S6/my8cNspk+BX3pSVq5K21Va7/aKK0su1Me2FVoTdJAgV/aKkmzYwZy2RmBXVMyJS1UpE3aKimzYzRgK2mmK/4U6mTt+Gsu7+drB19sy3uV6zPjvLvSOJJ6Cvwp08na8SNjee4/lK99YIucd+enO9/ZsfcXSQqlelKmk7Xj270fbrmkpJlEOk1X/ClTrXZ8s1NA5a/XihW3cSmnL3KBAn/KVArAF2UzTU0BRW2L2G7K6YtEU6onZSrVji8skmpeCihqW8RWMOBLN6+PPKe/uGldxVLKImmmK/6UKQbA8pTOJ/cejjy+1rz720aO8PXHjzPlTp8ZH7h6BZ8burJmGYZmWVYyF1+Lr0TisSTUTxscHPTR0dFOdyPVNu08EJmOGchleWx4c+Rzbhs5Ejk1cx60pU6+SipI2pnZIXcfrPd5SvUIUJhfX087wNcfPx7Z3uqgr12uRBqjVE+PmevMnEd/fKqudoCpDnxbLA7YisjcKfD3kEYWZ1Wb5llJn1nbg3/x/dq58Eyk1yjV00MaWZxVaXFTtUVPH7h6RX0dbIBFtLVr4ZlIr9EVfw+Zy1V70Y4tayPr0RcXPcXdMasVMvOMyQq1mZNU7VOkWyjw95BKi7PilCqoNiWyfDFWu6ZqFk2e94ppJZVhEKmfAn8PqXXVXkulevTtWoxVzZQ72UzfnM9NRC5Qjr+HDG0Y4PPvuZKBXLapUx7bfYUfpXguzT43kTTSFX8PiJrCCYW0zSf3HmbX/qOxp3WWv9aqSzqfSile2WuHLJHm0MrdLlc+hRMg02fgzBgQjbPKNeq1OiWXzXBmYlLlF0SqmOvK3ZpX/Gb2GuAHwMJw/H3ufruZrQa+AVwCHAL+yN1/Z2YLgbuBNwO/AG529+fr7ZjEEzWFMyofPzE5xR37nq66uKvT9fKLctkMh2+/rtPdEOlZcXL8vwU2u/s6YD3wB2a2EfgC8EV3fxNwGrglHH8LcDq0fzEcJy1Sz3TG8YlJ8uMTOBcWQI2MXdgRq5P18osyfcYd776i090Q6Wk1A78X/DrczYQfBzYD94X2PcBQuL013Cc8fq2ZRa2/kQaNjOWZ18B/2onJKT777afZ8OffY9XwQ03sWXyl3V+yKMOuG9cprSPSYrEGd82sj0I6503AXwE/Acbd/Vw45ARQ/Nc6ABwHcPdzZnaGQjro52WvuR3YDrBy5crGziKFivn4qLntUTn+Sjo5Y0fVNUU6I9Z0Tnefcvf1wHLgLcDljb6xu+9290F3H+zvr1wBUqJVysf3mbHrxnXset+6GVMfFy/om/0iHfbeN2uWjkgn1DWd093HzexR4K1Azszmh6v+5UAxWZwHVgAnzGw+cBGFQV5pokq5/fPu08G0+HtkLM+Oe59oW9/iqlb5U0Rap+YVv5n1m1ku3M4C7wCeAR4FbgyHbQMeDLf3hfuExw94EuaM9ph6iqrt2n80Vtqn3VRnR6Qz4qR6LgMeNbMngX8CHnb37wCfBj5lZsco5PDvCsffBVwS2j8FDDe/21Jp79zyEgYjY/lEzNaJojo7Ip1RM9Xj7k8CGyLan6OQ7y9v/w3wvqb0TioqpnH+6wNPcnaysOfVb85NMfrCL2ekeIo165NIdXZEOkO1errY6Au/nA76AO7wtYMvcttIIdgnZUFWlFw2o4FdkQ5R4O9SI2P5yI3OAe55vNCe1BRPNtOnRVoiHaQibV1mZCzPZ7/9dNX59+50bEFWlCWLMixaML/ufYBFpDUU+LtIkoqoxZXN9HH7u65QoBdJEAX+LpLknH1RLpth8UJd3YskmQJ/FyjWyE9qzr6omLtXoBdJNgX+BInaUGX0hV9yz8EXSd7yq5kGdHUv0jUU+NsoKrCXz7kvpnLy4xPsuPeJRK64LTWQy/LY8OZOd0NE6qDA3yaVAvtnv/0042cnmWc2q9JmkoK+waxvHdrsXKQ7aR5/m0TulHXeOX12EofI8spJ86Wb12uzc5EeoCv+Nkn6wGwty3JZbXYu0iN0xd8GpdsbJl1mnhU2cilhFP5wbdp5oGnnMjKWZ9POA6wefqiprysitemKvw127T/a6S7EkstmpkspFKePlub2i/v0Ag1d+UeNdzTjdUUkHl3xt0HS684P5LJ86eb1HL79uul0zmPDmxnIZWcN6E5MTjX8hyxqvKMZrysi8eiKvw2W5bKJzfE/v/Od07fLp5tW6nOjf8gqPT/pfyBFeoWu+Ntgx5a1zLPax7XbwvkXPv5i+iU/PoHDdJonSqMbqNSze5iINJ8CfxNVGrAc2jDARdlMh3s32/sGl0/fjkq/OMwK/s2Yux939zARaQ2lepqk1oDleJUyyp1Sutl5pTSLUxgDaGbRteLzK61iFpHWUuBvkloDllErczutNNhXyum3qiSD1gSIdI5SPU1S6Yq5eOWftKAPM3PqSr+IpIcCf5NUG5jsZA39JYsyfGjjyppBfWjDAJ9/z5UqySCSAjVTPWa2ArgbWEoh5bvb3b9sZhcDe4FVwPPATe5+2swM+DJwA3AW+Ii7/7A13W+9ahU1S11zeX/FPXA7oTxFM/iGi2ueh9IvIukQJ8d/DvjP7v5DM3sdcMjMHgY+Ajzi7jvNbBgYBj4NXA+sCT9XA3eG310n7grTkbE89x9KVsmBay7vn3FfQV1Eimqmetz9peIVu7v/M/AMMABsBfaEw/YAQ+H2VuBuLzgI5Mzssqb3vA3irjBN4paIpTN2RERK1ZXjN7NVwAbgcWCpu78UHnqZQioICn8Ujpc87URoK3+t7WY2amajp04lM0hVG7AtnaefxBWnSeyTiCRD7MBvZq8F7gc+4e6/Kn3M3Z3Z+3RU5e673X3Q3Qf7+/trP6EDqg3YFtM+I2P5RK44TeKCMRFJhliB38wyFIL+Pe7+QGh+pZjCCb9PhvY8sKLk6ctDW9eJmuJYqpj2Kc+nJ4ElsESEiCRDzcAfZuncBTzj7n9Z8tA+YFu4vQ14sKT9w1awEThTkhLqKqVTHCvJj0/wnSc6c3qLMpU/vtMJXCksIskQ54p/E/BHwGYzOxx+bgB2Au8ws2eBfxPuA3wXeA44BvwN8MfN73b7lJYormR8or1BdsmiDF+6eT0/+m/XV+yX0V0bwIhI+9Sczunu/4fZtbqKro043oGPNdivtqo2V7/4WPmmJJ1SPj9/x5a1fHLv4Vn9cgqzjTSFU0TKpb5WT7W5+sCMxzod9KNKKAxtGOATew9HHq+ZPSISJfWBv9Zc/U7Pzy9+yxiosmp4oEKBtSTONhKRzkt94E/yblBLFmW4/V1X1EzX7NiydsY3E1CBNRGpLPWBv1I54uLVcqe2TNz0exdzz79/a6xjVd9eROqR+sAfdbWcmWec/d05Tp+dbPuAbp8ZH7h6BZ8burKu56kWj4jElfrAX361fFE2w6sh6EP7gn4206cyyCLSFqkP/DDzannTzgNtm5ff7C0NRUTiUOAv065B3VZtaSgiUot24CrT7CmQa16/WFsaikiipDLwj4zl2bTzAKuHH5pRXhkKg73NrG/23Kmz2tJQRBIldameWrtqVVsJOxdT7ppxIyKJkror/ji7alUryFavPtVHFpGESV3gj7NSt5n19T9w9YraB4mItFHqAn+lwdtieyMbp695/eLpK/w+Mz60cWXdC7FERFotdTn+ait1Vw8/xDwzpry+ZVtm8MGrFeRFpDukJvCX1tzPLcqwcP48zkxMzlqpGyfoJ+FKvtoeAiIi1fRs4C8NjMXgPjlVCOqnz06SzfTxxZvXs2v/0bpX6iYh6FebmSQiUk1P5viLgTE/PoFT2BqxGPSLijN56l2p28wZP3MVZ2aSiEglPXnFHxUYoxTTJHFLL3dixW1USifJewiISPJ17RV/tdW3cQNgMZCWl1SI0mfW9hW35d9ciimd3KJM5PHacUtE4qgZ+M3sq2Z20syeKmm72MweNrNnw+8lod3M7CtmdszMnjSzq1rR6UoBsRj84wTA4tX70IaBGSUVlizKkJlns479i5vWtT1/Ximl447q/4jInMW54v8fwB+UtQ0Dj7j7GuCRcB/gemBN+NkO3Nmcbs5UK8d9zeX9NevtlF69D20Y4LHhzfx05zsZ+8x17HrfukTU1qn0zeXMxKTq/4jInNXM8bv7D8xsVVnzVuD3w+09wPeBT4f2u93dgYNmljOzy9z9pWZ1GKqvvi0uwKo2KXMgl60aJJNSW6fatpBJ6aOIdJ+55viXlgTzl4Gl4fYAcLzkuBOhramqrb6tNbDbTSmRqPGHbuq/iCRTw4O74eq+7h0KzWy7mY2a2eipU6fqem61gFhtYLfbUiLl4w/d1n8RSaa5Tud8pZjCMbPLgJOhPQ+UViVbHtpmcffdwG6AwcHBuv5wlO+TW7pyddf+o5HpkW7d8UopHRFptrkG/n3ANmBn+P1gSfufmNk3gKuBM83O7xdVCohRtXiUHhERuaBm4Dezr1MYyL3UzE4At1MI+N80s1uAF4CbwuHfBW4AjgFngY+2oM9VVfs2ICIiYF5nJcpWGBwc9NHR0U53Q0Skq5jZIXcfrPd5XbtyV0RE5kaBX0QkZRT4RURSRoFfRCRlFPhFRFImEbN6zOwUhWmh3eBS4Oed7kQb6Dx7i86ztxTP8w3u3l/vkxMR+LuJmY3OZfpUt9F59hadZ29p9DyV6hERSRkFfhGRlFHgr9/uTnegTXSevUXn2VsaOk/l+EVEUkZX/CIiKaPALyKSMgr8NZjZ82Z2xMwOm9loaLvYzB42s2fD7yWd7me9zOyrZnbSzJ4qaYs8Lyv4ipkdM7MnzeyqzvW8PhXO8w4zy4fP9LCZ3VDy2K3hPI+a2ZbO9Lo+ZrbCzB41sx+Z2dNm9vHQ3lOfZ5Xz7LXP8zVm9o9m9kQ4z8+G9tVm9ng4n71mtiC0Lwz3j4XHV9V8E3fXT5Uf4Hng0rK2/w4Mh9vDwBc63c85nNfbgauAp2qdF4U9Fv4XYMBG4PFO97/B87wD+NOIY/8l8ASwEFgN/ATo6/Q5xDjHy4Crwu3XAf8vnEtPfZ5VzrPXPk8DXhtuZ4DHw+f0TeD9of2vgf8Ybv8x8Nfh9vuBvbXeQ1f8c7MV2BNu7wGGOtiXOXH3HwC/LGuudF5bgbu94CCQC1tuJl6F86xkK/ANd/+tu/+UwoZCb2lZ55rE3V9y9x+G2/8MPAMM0GOfZ5XzrKRbP09391+Hu5nw48Bm4L7QXv55Fj/n+4BrzcyqvYcCf20OfM/MDpnZ9tC21C9sKfkysLQzXWu6Suc1ABwvOe4E1f/BdYM/CWmOr5ak6rr+PMPX/A0UrhJ79vMsO0/osc/TzPrM7DCF/cwfpvBtZdzdz4VDSs9l+jzD42eAS6q9vgJ/bW9z96uA64GPmdnbSx/0wvernpsT26vnFdwJ/B6wHngJ+IvOdqc5zOy1wP3AJ9z9V6WP9dLnGXGePfd5uvuUu68HllP4lnJ5M19fgb8Gd8+H3yeBb1H4EF4pfjUOv092rodNVem88sCKkuOWh7au5O6vhH9Y54G/4cLX/649TzPLUAiG97j7A6G55z7PqPPsxc+zyN3HgUeBt1JIyRX3SS89l+nzDI9fBPyi2usq8FdhZovN7HXF28B1wFPAPmBbOGwb8GBneth0lc5rH/DhMBtkI3CmJIXQdcry2f+WwmcKhfN8f5glsRpYA/xju/tXr5DPvQt4xt3/suShnjo5HLMAAADgSURBVPo8K51nD36e/WaWC7ezwDsojGc8CtwYDiv/PIuf843AgfANr7JOj2An+Qd4I4VZAU8ATwN/FtovAR4BngX+N3Bxp/s6h3P7OoWvxZMU8oW3VDovCrMM/opCnvEIMNjp/jd4nn8fzuPJ8I/mspLj/yyc51Hg+k73P+Y5vo1CGudJ4HD4uaHXPs8q59lrn+e/BsbC+TwFfCa0v5HCH65jwL3AwtD+mnD/WHj8jbXeQyUbRERSRqkeEZGUUeAXEUkZBX4RkZRR4BcRSRkFfhGRlFHgFxFJGQV+EZGU+f/Fx+hNjlukYQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot scores\n",
    "cfs_flat = list(chain(*cfs))\n",
    "bfs_flat = list(chain(*bfs))\n",
    "\n",
    "plt.scatter(cfs_flat, bfs_flat)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It looks like combining queries really won't do much. Seems that the scores from each sub-query are already pretty aligned and we can't benefit from learning the boost values which tell us how to combine scores."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
